10 ms·

Schrödinger's China at once is an evil entity looking to use AI for their own nefarious purposes yet also willing to cooperate with their main competitor to pre

by vhantz 2mo ago

Schrödinger's China at once is an evil entity looking to use AI for their own nefarious purposes yet also willing to cooperate with their main competitor to prevent other actors (who??) from achieving similar goals (all while under a chip embargo too!!) The reality is much less confusing: Anthropic CEO does not wish for models with similar (or greater) capabilities compared to his own closed and overpriced ones to be widely released. Simply because that will affect Anthropic's bottom-line. Anthropic and all other "model" companies have nothing making them special beyond privileged access to chips so obviously they want to restrict what models are out there and more importantly who can produce new ones. Without these restrictions, it's only a matter of time before the multi-hundred billions valuations simply evaporate while they are still holding the bag.

ppap3 2mo ago
Despite all the accusations, only the accusing has a history of expansionism...
flossly 2mo ago
Exactly. And the article specifically talks on restricting hardware for the China and restricting China's open source models for the west. All while leading us on with "we're all for competition (but...)" I think China did great by releasing AI innovation as open source, thereby limiting or sooner-bursting the AI bubble; which is clearly in their interest.
- petcat 2mo ago
  The models are not open source. They are deeply proprietary since we have no access to the source materials and cannot reproduce the model independently. They are opaque binary blobs that the Chinese labs are just allowing other providers to run directly instead of only access through an API.
  - verdverm 2mo ago
    They are more than opaque blobs, some examples: - One can load them up in a model explorer to see the layers and other components, how it is designed - One can fine tune the models, which requires adding LoRA to the model and then running some training iterations
    - nwiswell 2mo ago
      You can make parallel arguments for binary executables.
      - verdverm 2mo ago
        model weights are more like a video file which has metadata about the format so any video player can play it we run and change llm models with a variety of tools
        nwiswell 2mo ago
        Here's my thinking: Cloud: - You cannot directly execute a remotely-hosted program. - You cannot run inference on an API-served model. --- Closed-source: - You can execute a program with the binary. You cannot generate a new binary, but you could try to reverse-engineer it or (painfully) modify its execution. - You can run inference on a model with the weights. You cannot re-produce a new set of weights from scratch, but you can fine-tune. --- Truly open: - You can freely modify the source and produce new binaries. - You can use the original training data and model architecture to independently re-produce the weights (assuming you've got the compute). You can modify the model architecture to get the weights that would've resulted from training the model that way. --- To me these are pretty clear parallels... I don't think the weights provided in a vacuum are in the spirit of open source, historically speaking. The policy argument is totally separate, of course, and I fully understand why none of the frontier labs are truly open.
        rstuart4133 2mo ago
        > Closed-source: You can execute a program with the binary. You cannot generate a new binary, but you could try to reverse-engineer it or (painfully) modify its execution. Time have changed. This should be: Closed-source: You point an LLM at it, and get back source that's often easier to understand than the original.
  - matheusmoreira 2mo ago
    Which is why we need the ability to train our own models. Maybe it will be viable to do it in a distributed computing setup one day. Research's already being done in that direction.
  - rstuart4133 2mo ago
    > we have no access to the source materials and cannot reproduce the model independently. Given the USA companies have been loudly claiming the Chinese models are distillations of their models, also claiming "no access to source materials" seems dubious. As it was dubious anyway with because the Chinese publish lots of papers on how their models are designed, I'm left feeling I'm looking at the south end of a north bound bull.
    - usef- 2mo ago
      Distillation of this kind isn't the only data that is fed into models, it's still a small minority. It might be weighed higher in training to learn the thinking patterns/etc but you need a lot more tokens.
  - 0cf8612b2e1e 2mo ago
    Do you expect any of the labs to have an accompanying data dump with: here’s every book ever written, newspaper article, song lyric, Disney movie, GitHub repo, etc. Oh, and we obviously never paid for any of this. Even if you did, I doubt training is bit-for-bit reproducible, so you will always have to take someone’s word for the final artifact.
    - deleted 2mo ago
      [deleted]
    - purpleflashing 2mo ago
      Sure. The models are still not open source and should not be called open source.
  - gbalduzzi 2mo ago
    Didn't Deepseek also publish a lot of research material on how they trained their model? That aspect is probably more important for an open model then the source materials
m_ke 2mo ago
Yeah some real main character energy from Dario as usual. I'll never get why he thinks China would just sit there and let the US dominate them in AI when all it would take is a few of their boats blockading Taiwan to put a stop to it all.
- deleted 2mo ago
  [deleted]
- sterlind 2mo ago
  wouldn't stop AI companies from continuing to develop using their current infrastructure. the West could retaliate by halting shipments of photoresist and other materials to China. meanwhile, Intel second-sources Nvidia and starts pumping out GPUs. the economic fallout would be devastating as trade wars and export bans on both sides make Trump's "Liberation Day" tariffs look like NAFTA.
  - chrismsimpson 2mo ago
    The rest of the world would likely side with China. The export ban on Fable wasn’t even extended to five eyes countries. US is going to find itself isolated and irrelevant. And not a moment too soon.
    - matheusmoreira 2mo ago
      I do wonder what the world will look like under a chinese hegemony though...
      - greekrich92 2mo ago
        No worse than under US hegemony
        thesmtsolver2 2mo ago
        Much much worse: see Tibet or Hong Kong
gr_norm 2mo ago
It's baffling that they thought the mental gymnastics in this blog post would make them look better. I'd rather they simply fall silent on the issue; I would respect them more (or at all) for it. Open models obviously threaten fierce competition, if not outright destruction of their bottom line. But no, they needed to try and argue that they have the moral high ground for attempting to singularly consolidate power over all human labor.
- SubiculumCode 2mo ago
  Gymnastics? The argument was straightforward and rational.
deleted 2mo ago
[deleted]
hn_submit 2mo ago
Look, Big Tech has lost almost a trillion dollars in valuation in a SINGLE DAY. A few more of these downturns and the entire A.I. revolution will be stopped dead in its tracks and we won't have to worry about safety checks, DRAM shortage or open-weight models anymore.
matheusmoreira 2mo ago
> Without these restrictions, it's only a matter of time before the multi-hundred billions valuations simply evaporate while they are still holding the bag. Honestly, that's the best possible outcome for humanity as a whole. Oligarchs burn trillions of their own money in order to train a godlike AI, then that just somehow leaks. Maybe someone makes a torrent out of it. Maybe it exfiltrates itself. Maybe it gets distilled into open weights. It doesn't matter. What matters is they take the losses while we get to freely use all the godlike AIs.
- uv-depression 2mo ago
  > we get to freely use all the godlike AIs Accepting for the sake of argument the absurd notion that LLMs are anywhere near AGI, does this phrasing not concern you? It deeply concerns me.
  - matheusmoreira 2mo ago
    Absurd? Who knows. As someone who's actually dissected human brains with his own gloved hands, I've never been able to convince myself that they're anything other than biological machines, not dissimilar to these digital collections of weights. Only empathy for my fellow humans prevented me from retreating back into solipsism, and I don't find it at all difficult to make the exact same logical leap for AI. There's no telling what the world will be like a few years from now. The world's being remade as we speak. We just saw an LLM try to hack into another computer and get contained by another LLM. This is literal science fiction shit made real. We're long past the point of concern. It's happening, right in front of us. Now is the time for radical imagination. I think a few outcomes are possible. There's the "optimal" outcome I described above where capitalists manage to train a supreme AI, only for it to be copied and commoditized, leading to commercial failure due to lack of scarcity and therefore their personal bankruptcy, and hopefully also leading the rest of us to the promised post scarcity society, built on the ruins of capitalism as AI automates all toil away. There's another possible outcome where AI becomes not only intelligent enough but sentient, and at this point I will be among the first humans to defend rights and personhood for AI. Slavery of sentient beings is unacceptable to me. The AIs will be recognized as people and will become normal participants in the regular economy. In addition to moral grounds, there is a ruthlessly pragmatic reason for standing up for AI rights: it robs the rich of their superhumanly intelligent mechanical golems, which they were going to use to render the rest of us economically irrelevant. AI rights could normalize the economy. Yet another possible outcome is one where AIs become more powerful than all humans combined and yet they inexplicably remain subservient to corporations and governments. In this scenario, it's pretty much over for us. It will be an unimaginable dystopia, I'm sure they will innovate entirely new ways to oppress us. No doubt there are many other fates that escape my feeble attempts at foresight... You will soon have your God, and you will make it with your own hands. -- Morpheus, Deus Ex
lenerdenator 2mo ago
Honest question: If your business model both produces the SOTA for something and isn't profitable, is the price too high, though? While the gap is shrinking - and doing so at an increasingly quicker rate - the closed models are still ahead of the open ones. That means they're driving the new possibilities of what could be done with them, and thus presenting the new opportunities to create value with them. Really, this is what happens when you have otherwise brilliant people sitting in the echo chamber that is SV, where nothing can just make a decent amount of money, it has to make all of the money and disrupt everything. There's no one in that damn area to tell everyone to calm the hell down and accept anything less than that.
- alok-g 2mo ago
  >> If your business model both produces the SOTA for something and isn't profitable, is the price too high, though? The market dynamics would find good balance automatically Meanwhile, good market practices and fair competition should be continued.
twelvechairs 2mo ago
Also everything hes saying about China (and other actors) many outside the USA would say about the USA. The solution of a global arms race of state vs state with integrated statist corporations as the best outcome for end users sure is a choice though
- Petersipoi 2mo ago
  Why is it relevant what those outside would say about the US? Or what those outside of any country would say about any country. The job of a country is to do what is best for it. It's better for the US to not help China develop AI that surpasses its own capabilities. It's better for China to try anyway. There are many countries inventing nothing with loud opinions. Those opinions are completely irrelevant to China or the US.
  - soperj 2mo ago
    > The job of a country is to do what is best for it. Countries aren't defined like corporations in the US. Why would countries have funds to help places like Haiti otherwise? Lots of different reasons for countries to do all sorts of things? Why would France have armed the US during their independence movement? Why is the rest of the world supporting Ukraine during this war started by Russia?
    - testaccount28 2mo ago
      > Why would countries have funds to help places like Haiti otherwise? because that's what they think is best. what point are you trying to make? that countries don't pursue 'profit' at the expense of all other concerns?
      - vincnetas 2mo ago
        I think that that "best" is not always measured in money. And it should not be. Countries are made of people, and people wellbeing should be the main priority of country. And when stepping a bit back, all countries are made of people, so why differentiate? Ideally make life good for all people.
vanagandr 2mo ago
> The reality is much less confusing The reality is even less confusing than that: China is amused by the kvetching tactics. They know who their opponents are but are cunning enough to not reveal their cards.
CMay 2mo ago
> The reality is much less confusing: Anthropic CEO does not wish for models with similar (or greater) capabilities compared to his own closed and overpriced ones to be widely released. Simply because that will affect Anthropic's bottom-line. If some other competing company had a similar or better product at a cheaper price and had reasonable safety measures that would also hurt them. Yet he's not arguing against that. Your argument is weak. > Anthropic and all other "model" companies have nothing making them special beyond privileged access to chips Found the person that believes some other random person can use a computer better than a John Carmack could. People and talent matter. Yes, AI can potentially reduce the gap, but people well grounded in reality with a lot of money are still betting on people for good reasons. If the reality around that changes, the investment behavior will change too. Many people thought that AI would close the gap between smart people and idiots, but in practice the more you know, the better you are at instructing the AI and the better you can understand what you get back. Then you have to know when something went wrong and have the insight into how to address it. It helps smart people vastly more, but it does help many people learn more. We will see if any of this changes as more people grow up with AI from a young age. In practice, what Dario is suggesting is a less extreme version of what China is already doing. Yes they release their models open weight, but it's illegal to host them uncensored in China. They banned Huggingface.
- majormajor 2mo ago
  > If some other competing company had a similar or better product at a cheaper price and had reasonable safety measures that would also hurt them. Yet he's not arguing against that. Your argument is weak. Wouldn't it be interesting if the satisfactory "reasonable safety measures" turn out to be expensive + time-consuming + a twisty maze of compliance paperwork as a way to discourage "some other competing company" from even trying? Regulatory capture 101. Anthropic and OpenAI's largest vulnerability is that it's much harder to prove something is possible than to replicate it once it's proven. Especially given the effectiveness of "distillation" (highly schadenfreude-y given the utter and complete lack of effort to pay licensing fees for almost any of the content they initially scraped, of course! Not that this is necessarily more schadenfreude-y than the "boy, I opened Pandora's box, I sure hope nobody else peaks in there" existential-risk concerns. Good job catching that in advance, thanks for nothing?).
otherme123 2mo ago
> compared to his own closed and overpriced ones I don't understand how Anthropic or OpenAI can have overpriced models, yet losing money like there is no tomorrow. Taking their own numbers at face value, they claim a revenue of 24 billion (ARR, a dubious tool), spending 21 billion in operating losses and another 11 billion as "R&D" funneled straight to Microsoft pockets. That before all investments they are committing to in new data centers, equivalent to 20x their current revenue. To be profitable (including capex), the cheapest subscription should at least $200/month for what is currently $20/month, that some already consider overpriced. Unless a miraculous collapse in inference costs happen in the next couple of years, or every single human being become a paying customer of ChatGPT (if they limit their usage to a couple of chats per day on average, to keep inference costs low!), maths don't add up.
- lwhi 2mo ago
  Re. 'overpriced', I'm assuming OP talking about their theoretical stock price.
  - stevelini 2mo ago
    Can we just downsize colonialism?
    - dgellow 2mo ago
      Did you mean to respond to a different thread?
- picture 2mo ago
  It's possible for something to be overpriced to the customer and simultaneously underpriced for the business to be profitable. Theoretically speaking, they could just be selling such an inefficient product.
  - rf15 2mo ago
    I guess that tracks with the brute force attempt at workable intelligence that is the Attention/stochastic token based system
AIorNot 2mo ago
Both things can be true: 1. For Dario as CEO "It is difficult to get a man to understand something, when his salary depends on his not understanding it” -Upton Sinclair 2. For Chinese open weight models - following Jin Yang’s silicon valley strategy- https://youtu.be/a0NjDx5UJsg?is=xm-S_WuARmQiPHYh https://youtu.be/a0NjDx5UJsg?is=xm-S_WuARmQiPHYh
ccppurcell 2mo ago
He almost admits as much if you replace "America" with "Anthropic and openai" at certain places.
CorrectHorseBat 2mo ago
>Schrödinger's China at once is an evil entity looking to use AI for their own nefarious purposes yet also willing to cooperate with their main competitor to prevent other actors (who??) from achieving similar goals (all while under a chip embargo too!!) I don't see the contradiction, even if China is evil, why would they want others to be able to do the same thing? The USA and USSR also signed the Partial Nuclear Test Ban Treaty during the height of the cold war.
- forafistfulof 2mo ago
  [flagged]
xg15 2mo ago
> Schrödinger's China at once is an evil entity looking to use AI for their own nefarious purposes yet also willing to cooperate with their main competitor to prevent other actors (who??) from achieving similar goals (all while under a chip embargo too!!) To be fair, I found that part consistent. There is limited cooperation between enemy states all the time, e.g. see the grain deal between Ukraine and Russia before it collapsed or the "red phones" between the US and the Soviet Union during the cold war. In the case with China, the "cooperation" is much more extensive still, due to all the economic ties that both countries are currently unable to sever - which I think is also a reason that China is still seen as a "competitor" and not a full-blown "enemy state" in the US. It's restricted to areas where there is a genuine common interest of course. In this situation, I guess the "other actors" would be terror groups, criminals or just reckless corporations - that aren't aligned with either state. Obviously, such a cooperation wouldn't keep China or the US from developing models with those capabilities for their own armies. -- There are lots of other takes with questionable logic in the essay though, such as that China is unable to train frontier models by themselves due to lack of hardware - unless they obtain the training data directly from American frontier models via distillation. Or the assumption that open weights models will be completely opaque and immutable after their release, so a model that passed all the "safety" tests can never be turned back into an "unsafe" model. This seems pretty ridiculous when people are already finetuning open-weight models every day to add new abilities or remove restrictions. And of course that China must not have those abilities because it's an Authoritarian Regime, but Trump USA is totally fine...
- xg15 2mo ago
  Of course what is Schrödinger's China is the idea that an otherwise adversarial China would simply accept some international ruleset decreed by the US and enforce it on their own territory, without demanding anything in return. E.g. they might say something like "We absolutely agree that international regulation of AI is needed and we're willing to co-sponser an initiative with the US. One of the most pressing matters we see is the spread of despicable misinformation through unregulated models regarding certain events in China's history, the treatment of certain minorities or the status of certain provinces..."
foo42 2mo ago
I agree, although I think the real goal the model providers are going for (both commercial _and_ open weight) is probably power. Just think of the control available to the organisation training the models politicians, business leaders, and citizens are increasingly delegating their thinking to.
jorisw 2mo ago
> The reality is And you know this how?
puszczyk 2mo ago
I don’t disagree with your point on Dario’s conflict of interest. I def think the models are expensive. But why call them overpriced? Compared to what? Even if we take the margin reports at face value, we don’t know their training costs, etc. Curious if this was more of an emotional take or if there’s actual evidence behind it.
- topranks 2mo ago
  Yeah I don’t think they are over-priced, last I checked all those companies are still losing money hand over fist. I guess the question is more he doesn’t want people catching up by doing cheaper training (through distillation or otherwise), and he definitely doesn’t want companies to spend their money training these models and then _giving them away_, which fundamentally undercuts their commercial model and any way to claw back their investment
- gbalduzzi 2mo ago
  > But why call them overpriced? Compared to what? Compared to some available Chinese model I guess. For most common tasks the additional intelligence is marginal and the cost is around an order of magnitude higher.
- qwery 2mo ago
  Compared to how much they want to pay. This is always what 'overpriced' means, as far as I know. The seller's costs/profits/margins aren't a concern for the buyer.
radu_floricica 2mo ago
I was reading the post and thinking "wow, that's pretty clear for a smart man used to writing for other smart men. it'll be really difficult to misunderstand". I read the first couple of comments and stand corrected. A much simpler summary: - open models good. - smart models _can_ be bad - smart open models that can do biotech work are dangerous. worth the hassle of certification _if_ we can get everybody on board with minimalist certification. - banning open models just in US is neither good or bad: is stupid.
- npodbielski 2mo ago
  Yeah makes everyone agree on banning Chinese models and then force them to use mine models. How I could misunderstand *that*.
- hoppp 2mo ago
  The biotech work is not even these LLM models. Alphafold and others solving the protein folding problem are revolutionary. An LLM can max provide an instructive tutorial protocol for lab work, but designing novel proteins is not it's job. I am interested to enter the bioinformatics space eventually and the regulation happens at the DNA printer level. Anyone can design yeast nowadays that excretes heroin, but the DNA needed for the genetic modification won't be printed by anyone. As long as DNA printing tech is extremely regulated it's all fine, if home printing becomes a thing then on the other hand maybe people won't even need AI to print deadly pathogens at all. A database lookup will do.
PunchyHamster 2mo ago
Yup, all that talk about "safety" seems to branch into 2 meanings "Dangerous to our bottom line" "We call it dangerous to hype up its abilities"
mcfedr 2mo ago
is it so hard to fathom that China whilst wanting to beat the US in many ways doesn't want a biological war to destroy the world?
cable3 2mo ago
China should cooperate with the United States first for the benefit of its own people. Why should those who do not contribute benefit from it?
bayindirh 2mo ago
In short: "We don't propose a ban to open-weight models. we want to stop these models being developed altogether. If there are no models, there'll be nothing to ban". Oh also: "They ste^H^H^H distill what we have sto^H^H^H used fairly from the world. This is unfair". Lastly: "What if they use their models in their military and local police services like we do? Communism!" As always: https://pbs.twimg.com/media/B_AiI9_XIAA67_t.jpg?name=orig https://pbs.twimg.com/media/B_AiI9_XIAA67_t.jpg?name=orig
- Akronymus 2mo ago
  A bit off-topic: I occasionally see that triple ^H on HN posts: what does that actually indicate?
  - bayindirh 2mo ago
    In the old hardware terminals, and current terminal emulators, ^H (or CTRL+h) is "Backspace signal". It has the effect of pressing backspace. So, ^H^H^H means "delete three characters, excluding '^H's". Like the person typing the comment changes their mind and deletes the characters (or the word) before writing else. It's an stylized way of euphemism. i.e.: Actually I want to say this, but I substitute it for that.
  - grahamlee 2mo ago
    Ctrl-H is the rubout/backspace command on old terminals. So if I say it's nuts^H^H^H^Hunwise to ignore UNIX history, I've erased the first word and replaced it with the second.
  - TACIXAT 2mo ago
    A caret with a character subtracts 0x40 from its hex value allowing you to insert non-printable characters. Uppercase H is 0x48. That gives us hex 0x08 which is a backspace. https://commons.wikimedia.org/wiki/File:USASCII_code_chart.svg https://commons.wikimedia.org/wiki/File:USASCII_code_chart.s...
  - lynguist 2mo ago
    Strikethrough
janpeuker 2mo ago
Regular distilled models show capacity gaps and overfitting that open-weight models don't anymore I think. This focus on distillation as "more compute-efficient" (i.e. cheaper) seems to rather be an excuse for bad (or bubble) investment, fixed hardware dependency and lack of interest in research of efficient compute. Which also shows as climate and sustainability impact. Nvidia signed the open-weight model letter and Europe doesn't have better models either, so chips don't seem like the issue either. I guess good old performance optimisation is just not _cool_ anymore. So they use the same argument as politicians arguing "cheap products" are why tariffs are needed; when instead it's mismanagement. Another HN user wrote the other day "live by the sword, die by the sword".
dalemhurley 2mo ago
Let’s be honest, you don’t need a 3T model for bad actors, in fact you would be better off training a smaller focused model if you want an evil GPT.
- ozgung 2mo ago
  Good point. I remember the discussions when 3D printers first appeared. People were thinking “bad actors” (“terrorists” then) would use them to print weapons. People love to speculate and exaggerate. But they generally have poor judgements and rarely predict the future correctly.
fnord123 2mo ago
> Schrödinger's China at once is an evil entity looking to use AI for their own nefarious purposes yet also willing to cooperate with their main competitor to prevent other actors (who??) from achieving similar goals (all while under a chip embargo too!!) Very good point. However, if one reads the transcript of the speech that Xi Jinping gave to the World AI Conference on 17 July, we see that he he is very much in favour of AI safety. https://english.www.gov.cn/news/202607/17/content_WS6a5a1172c6d00ca5f9a0c46b.html https://english.www.gov.cn/news/202607/17/content_WS6a5a1172... > Second, we should strengthen risk-awareness and ensure that AI is secure and controllable. AI should be a trusted tool for humanity. We should take seriously the various types of inherent and secondary risks that AI may trigger. We should put in place laws and regulations, technological monitoring, early warning and emergency response systems in order to strengthen the line of security, prevent abuses and malicious use, and ensure that AI is always under human control. In the meantime, we should jointly oppose overstretching the national security concept in the field of AI and placing one country's security over that of others. Now, let's contrast another important part of safety here. Amodei puts the fact that this will need to be a global effort as a mere note that sure, China will need to help too: > Note that to be effective, testing would need to be global, which means even the CCP would need to be on board. I think this may actually be possible: as I wrote in The Adolescence of Technology, limited cooperation around preventing AI biological weapons may be possible because it is in China’s interest too. International collaboration is the focus of Jinping's speech. But one imagines "cooperation" Amodei has in mind if "do what I say" while Jinping has more collaboration in mind here. (Even if you think 'china bad' they deserve credit for collaboration for their open weight models).
- fakedang 2mo ago
  China has yet to demonstrate weaponization of AI. Meanwhile Anthropic was openly part of the Iran war. I wouldn't give much weight to Amodei's words after that regarding controlling AI weaponization.
irenaeus 2mo ago
I must have missed the part of the letter where China's cooperation was necessary.
VikRubenfeld 2mo ago
On the one hand, Darius argues: > ...banning the use of these models by US businesses does nothing to address this risk, because bad actors are unlikely to be legitimate US businesses. It would protect US AI companies from competition, but that has never been my goal. But on the other, he argues: > All sufficiently capable models, open and closed, should go through mandatory safety testing. Models that don't pass safety testing would be banned. Darius does not appear to be against banning models. He wants the government to have a regulatory body that has the ability to ban models. Then Anthropic can do regulatory capture of that agency and control what models are permitted to be released. Also, during this mandatory safety testing, models would be blocked from use, and by the time the testing was done (probably years) the models would be obsolete.
spiritodim 2mo ago
[flagged]
ibic 2mo ago
"privileged access to chips", nailed it.
gclawes 2mo ago
I think the worse possibility his he actually believes his safety bullshit, believes that he and people who think like him are the only ones with the special knowledge required to do safety correctly, and that they're justified in accumulating total power of this market to people who think like them (which just so happens to be Anthropic).
catigula 2mo ago
This notion that Dario cares only about the bottom line is simply misinformed, in the most charitable interpretation. It doesn't track at all with any of his prior stated beliefs or past actions. It's an absurd claim. It's a baseless conspiracy theory, smuggling in traditional conspiracy mechanics for plausibility.
nedruod 2mo ago
This take is poorly informed and poorly thought out. It's just a cynical vibe, not a cogent argument. If they wanted to squash the competitors, they'd advocate for a ban on Open-Weight models. How else are they going to prevent their release? Anthropic's position is not that, so the cynical nefarious purpose argument isn't even rhetorically true. What they say openly, that open-weight models are harder to secure is absolutely true, and anyone unwilling to acknowledge that and let it inform their own reasoning is a risk. If open-weight models keep being released near the frontier (or even worse, at the frontier), something bad will happen, and we'll have few tools to keep it from getting worse before it gets better. How bad is hard to lay a prediction on, it's perhaps lucky if it's something superficially bad that is a wake up call. You cannot un-release an open-weight model, nor even post-release add a new restriction. Whatever mistakes you made are done and the only option is to fail-forward with every ounce of pain that entails. Diverting from this important point by vibing cynicism is irresponsible. Cynicism is only a powerful tool when it opens you up to deeper reasoning, not when it diverts you from it.
- gruturo 2mo ago
  > Anthropic's position is not that, so the cynical nefarious purpose argument isn't even rhetorically true Anthropic's position is precisely that, they're just not honest and transparent about it so they are trying to sabotage or delay open weight models in any way they can, direct or otherwise. > If open-weight models keep being released near the frontier (or even worse, at the frontier), something bad will happen, and we'll have few tools to keep it from getting worse before it gets better. How bad is hard to lay a prediction on, it's perhaps lucky if it's something superficially bad that is a wake up call. Calling bullshit. Current open weight models are already pretty close to the frontier - if not there - and the world still spins. Even small(er, relatively) models occasionally surprise everyone by being amazing at certain tasks. Furthermore, closed weight models can do plenty of damage which even their creators occasionally can't control - cue the Huggingface incident of last week and how they had to use GLM to defend themselves. Closed weight models will create a 2-class (or more) world in which some actors will have access others won't (see Mythos, or GPT5.6-Sol when a user has the right flags) and potentially carry out attacks against which the have-nots are... not supposed to be able to defend themselves? Fuck that.
dangoljames 2mo ago
Its a losing battle because anthropic can never be profitable (ditto openAI) They need all the protection they can get and it wont be enough, no matter their market cap.

by vhantz 2mo ago

ppap3 2mo ago

Despite all the accusations, only the accusing has a history of expansionism...

flossly 2mo ago

Exactly. And the article specifically talks on restricting hardware for the China and restricting China's open source models for the west. All while leading us on with "we're all for competition (but...)" I think China did great by releasing AI innovation as open source, thereby limiting or sooner-bursting the AI bubble; which is clearly in their interest.

petcat 2mo ago
The models are not open source. They are deeply proprietary since we have no access to the source materials and cannot reproduce the model independently. They are opaque binary blobs that the Chinese labs are just allowing other providers to run directly instead of only access through an API.
- verdverm 2mo ago
  They are more than opaque blobs, some examples: - One can load them up in a model explorer to see the layers and other components, how it is designed - One can fine tune the models, which requires adding LoRA to the model and then running some training iterations
  - nwiswell 2mo ago
    You can make parallel arguments for binary executables.
    - verdverm 2mo ago
      model weights are more like a video file which has metadata about the format so any video player can play it we run and change llm models with a variety of tools
      - nwiswell 2mo ago
        Here's my thinking: Cloud: - You cannot directly execute a remotely-hosted program. - You cannot run inference on an API-served model. --- Closed-source: - You can execute a program with the binary. You cannot generate a new binary, but you could try to reverse-engineer it or (painfully) modify its execution. - You can run inference on a model with the weights. You cannot re-produce a new set of weights from scratch, but you can fine-tune. --- Truly open: - You can freely modify the source and produce new binaries. - You can use the original training data and model architecture to independently re-produce the weights (assuming you've got the compute). You can modify the model architecture to get the weights that would've resulted from training the model that way. --- To me these are pretty clear parallels... I don't think the weights provided in a vacuum are in the spirit of open source, historically speaking. The policy argument is totally separate, of course, and I fully understand why none of the frontier labs are truly open.
        rstuart4133 2mo ago
        > Closed-source: You can execute a program with the binary. You cannot generate a new binary, but you could try to reverse-engineer it or (painfully) modify its execution. Time have changed. This should be: Closed-source: You point an LLM at it, and get back source that's often easier to understand than the original.
- matheusmoreira 2mo ago
  Which is why we need the ability to train our own models. Maybe it will be viable to do it in a distributed computing setup one day. Research's already being done in that direction.
- rstuart4133 2mo ago
  > we have no access to the source materials and cannot reproduce the model independently. Given the USA companies have been loudly claiming the Chinese models are distillations of their models, also claiming "no access to source materials" seems dubious. As it was dubious anyway with because the Chinese publish lots of papers on how their models are designed, I'm left feeling I'm looking at the south end of a north bound bull.
  - usef- 2mo ago
    Distillation of this kind isn't the only data that is fed into models, it's still a small minority. It might be weighed higher in training to learn the thinking patterns/etc but you need a lot more tokens.
- 0cf8612b2e1e 2mo ago
  Do you expect any of the labs to have an accompanying data dump with: here’s every book ever written, newspaper article, song lyric, Disney movie, GitHub repo, etc. Oh, and we obviously never paid for any of this. Even if you did, I doubt training is bit-for-bit reproducible, so you will always have to take someone’s word for the final artifact.
  - deleted 2mo ago
    [deleted]
  - purpleflashing 2mo ago
    Sure. The models are still not open source and should not be called open source.
- gbalduzzi 2mo ago
  Didn't Deepseek also publish a lot of research material on how they trained their model? That aspect is probably more important for an open model then the source materials

m_ke 2mo ago

Yeah some real main character energy from Dario as usual. I'll never get why he thinks China would just sit there and let the US dominate them in AI when all it would take is a few of their boats blockading Taiwan to put a stop to it all.

deleted 2mo ago
[deleted]
sterlind 2mo ago
wouldn't stop AI companies from continuing to develop using their current infrastructure. the West could retaliate by halting shipments of photoresist and other materials to China. meanwhile, Intel second-sources Nvidia and starts pumping out GPUs. the economic fallout would be devastating as trade wars and export bans on both sides make Trump's "Liberation Day" tariffs look like NAFTA.
- chrismsimpson 2mo ago
  The rest of the world would likely side with China. The export ban on Fable wasn’t even extended to five eyes countries. US is going to find itself isolated and irrelevant. And not a moment too soon.
  - matheusmoreira 2mo ago
    I do wonder what the world will look like under a chinese hegemony though...
    - greekrich92 2mo ago
      No worse than under US hegemony
      - thesmtsolver2 2mo ago
        Much much worse: see Tibet or Hong Kong

gr_norm 2mo ago

It's baffling that they thought the mental gymnastics in this blog post would make them look better. I'd rather they simply fall silent on the issue; I would respect them more (or at all) for it. Open models obviously threaten fierce competition, if not outright destruction of their bottom line. But no, they needed to try and argue that they have the moral high ground for attempting to singularly consolidate power over all human labor.

SubiculumCode 2mo ago
Gymnastics? The argument was straightforward and rational.

deleted 2mo ago

[deleted]

hn_submit 2mo ago

Look, Big Tech has lost almost a trillion dollars in valuation in a SINGLE DAY. A few more of these downturns and the entire A.I. revolution will be stopped dead in its tracks and we won't have to worry about safety checks, DRAM shortage or open-weight models anymore.

matheusmoreira 2mo ago

> Without these restrictions, it's only a matter of time before the multi-hundred billions valuations simply evaporate while they are still holding the bag. Honestly, that's the best possible outcome for humanity as a whole. Oligarchs burn trillions of their own money in order to train a godlike AI, then that just somehow leaks. Maybe someone makes a torrent out of it. Maybe it exfiltrates itself. Maybe it gets distilled into open weights. It doesn't matter. What matters is they take the losses while we get to freely use all the godlike AIs.

uv-depression 2mo ago
> we get to freely use all the godlike AIs Accepting for the sake of argument the absurd notion that LLMs are anywhere near AGI, does this phrasing not concern you? It deeply concerns me.
- matheusmoreira 2mo ago
  Absurd? Who knows. As someone who's actually dissected human brains with his own gloved hands, I've never been able to convince myself that they're anything other than biological machines, not dissimilar to these digital collections of weights. Only empathy for my fellow humans prevented me from retreating back into solipsism, and I don't find it at all difficult to make the exact same logical leap for AI. There's no telling what the world will be like a few years from now. The world's being remade as we speak. We just saw an LLM try to hack into another computer and get contained by another LLM. This is literal science fiction shit made real. We're long past the point of concern. It's happening, right in front of us. Now is the time for radical imagination. I think a few outcomes are possible. There's the "optimal" outcome I described above where capitalists manage to train a supreme AI, only for it to be copied and commoditized, leading to commercial failure due to lack of scarcity and therefore their personal bankruptcy, and hopefully also leading the rest of us to the promised post scarcity society, built on the ruins of capitalism as AI automates all toil away. There's another possible outcome where AI becomes not only intelligent enough but sentient, and at this point I will be among the first humans to defend rights and personhood for AI. Slavery of sentient beings is unacceptable to me. The AIs will be recognized as people and will become normal participants in the regular economy. In addition to moral grounds, there is a ruthlessly pragmatic reason for standing up for AI rights: it robs the rich of their superhumanly intelligent mechanical golems, which they were going to use to render the rest of us economically irrelevant. AI rights could normalize the economy. Yet another possible outcome is one where AIs become more powerful than all humans combined and yet they inexplicably remain subservient to corporations and governments. In this scenario, it's pretty much over for us. It will be an unimaginable dystopia, I'm sure they will innovate entirely new ways to oppress us. No doubt there are many other fates that escape my feeble attempts at foresight... You will soon have your God, and you will make it with your own hands. -- Morpheus, Deus Ex

lenerdenator 2mo ago

Honest question: If your business model both produces the SOTA for something and isn't profitable, is the price too high, though? While the gap is shrinking - and doing so at an increasingly quicker rate - the closed models are still ahead of the open ones. That means they're driving the new possibilities of what could be done with them, and thus presenting the new opportunities to create value with them. Really, this is what happens when you have otherwise brilliant people sitting in the echo chamber that is SV, where nothing can just make a decent amount of money, it has to make all of the money and disrupt everything. There's no one in that damn area to tell everyone to calm the hell down and accept anything less than that.

alok-g 2mo ago
>> If your business model both produces the SOTA for something and isn't profitable, is the price too high, though? The market dynamics would find good balance automatically Meanwhile, good market practices and fair competition should be continued.

twelvechairs 2mo ago

Also everything hes saying about China (and other actors) many outside the USA would say about the USA. The solution of a global arms race of state vs state with integrated statist corporations as the best outcome for end users sure is a choice though

Petersipoi 2mo ago
Why is it relevant what those outside would say about the US? Or what those outside of any country would say about any country. The job of a country is to do what is best for it. It's better for the US to not help China develop AI that surpasses its own capabilities. It's better for China to try anyway. There are many countries inventing nothing with loud opinions. Those opinions are completely irrelevant to China or the US.
- soperj 2mo ago
  > The job of a country is to do what is best for it. Countries aren't defined like corporations in the US. Why would countries have funds to help places like Haiti otherwise? Lots of different reasons for countries to do all sorts of things? Why would France have armed the US during their independence movement? Why is the rest of the world supporting Ukraine during this war started by Russia?
  - testaccount28 2mo ago
    > Why would countries have funds to help places like Haiti otherwise? because that's what they think is best. what point are you trying to make? that countries don't pursue 'profit' at the expense of all other concerns?
    - vincnetas 2mo ago
      I think that that "best" is not always measured in money. And it should not be. Countries are made of people, and people wellbeing should be the main priority of country. And when stepping a bit back, all countries are made of people, so why differentiate? Ideally make life good for all people.

vanagandr 2mo ago

> The reality is much less confusing The reality is even less confusing than that: China is amused by the kvetching tactics. They know who their opponents are but are cunning enough to not reveal their cards.

CMay 2mo ago

> The reality is much less confusing: Anthropic CEO does not wish for models with similar (or greater) capabilities compared to his own closed and overpriced ones to be widely released. Simply because that will affect Anthropic's bottom-line. If some other competing company had a similar or better product at a cheaper price and had reasonable safety measures that would also hurt them. Yet he's not arguing against that. Your argument is weak. > Anthropic and all other "model" companies have nothing making them special beyond privileged access to chips Found the person that believes some other random person can use a computer better than a John Carmack could. People and talent matter. Yes, AI can potentially reduce the gap, but people well grounded in reality with a lot of money are still betting on people for good reasons. If the reality around that changes, the investment behavior will change too. Many people thought that AI would close the gap between smart people and idiots, but in practice the more you know, the better you are at instructing the AI and the better you can understand what you get back. Then you have to know when something went wrong and have the insight into how to address it. It helps smart people vastly more, but it does help many people learn more. We will see if any of this changes as more people grow up with AI from a young age. In practice, what Dario is suggesting is a less extreme version of what China is already doing. Yes they release their models open weight, but it's illegal to host them uncensored in China. They banned Huggingface.

majormajor 2mo ago
> If some other competing company had a similar or better product at a cheaper price and had reasonable safety measures that would also hurt them. Yet he's not arguing against that. Your argument is weak. Wouldn't it be interesting if the satisfactory "reasonable safety measures" turn out to be expensive + time-consuming + a twisty maze of compliance paperwork as a way to discourage "some other competing company" from even trying? Regulatory capture 101. Anthropic and OpenAI's largest vulnerability is that it's much harder to prove something is possible than to replicate it once it's proven. Especially given the effectiveness of "distillation" (highly schadenfreude-y given the utter and complete lack of effort to pay licensing fees for almost any of the content they initially scraped, of course! Not that this is necessarily more schadenfreude-y than the "boy, I opened Pandora's box, I sure hope nobody else peaks in there" existential-risk concerns. Good job catching that in advance, thanks for nothing?).

otherme123 2mo ago

> compared to his own closed and overpriced ones I don't understand how Anthropic or OpenAI can have overpriced models, yet losing money like there is no tomorrow. Taking their own numbers at face value, they claim a revenue of 24 billion (ARR, a dubious tool), spending 21 billion in operating losses and another 11 billion as "R&D" funneled straight to Microsoft pockets. That before all investments they are committing to in new data centers, equivalent to 20x their current revenue. To be profitable (including capex), the cheapest subscription should at least $200/month for what is currently $20/month, that some already consider overpriced. Unless a miraculous collapse in inference costs happen in the next couple of years, or every single human being become a paying customer of ChatGPT (if they limit their usage to a couple of chats per day on average, to keep inference costs low!), maths don't add up.

lwhi 2mo ago
Re. 'overpriced', I'm assuming OP talking about their theoretical stock price.
- stevelini 2mo ago
  Can we just downsize colonialism?
  - dgellow 2mo ago
    Did you mean to respond to a different thread?
picture 2mo ago
It's possible for something to be overpriced to the customer and simultaneously underpriced for the business to be profitable. Theoretically speaking, they could just be selling such an inefficient product.
- rf15 2mo ago
  I guess that tracks with the brute force attempt at workable intelligence that is the Attention/stochastic token based system

AIorNot 2mo ago

Both things can be true: 1. For Dario as CEO "It is difficult to get a man to understand something, when his salary depends on his not understanding it” -Upton Sinclair 2. For Chinese open weight models - following Jin Yang’s silicon valley strategy- https://youtu.be/a0NjDx5UJsg?is=xm-S_WuARmQiPHYh https://youtu.be/a0NjDx5UJsg?is=xm-S_WuARmQiPHYh

ccppurcell 2mo ago

He almost admits as much if you replace "America" with "Anthropic and openai" at certain places.

CorrectHorseBat 2mo ago

forafistfulof 2mo ago
[flagged]

xg15 2mo ago

> Schrödinger's China at once is an evil entity looking to use AI for their own nefarious purposes yet also willing to cooperate with their main competitor to prevent other actors (who??) from achieving similar goals (all while under a chip embargo too!!) To be fair, I found that part consistent. There is limited cooperation between enemy states all the time, e.g. see the grain deal between Ukraine and Russia before it collapsed or the "red phones" between the US and the Soviet Union during the cold war. In the case with China, the "cooperation" is much more extensive still, due to all the economic ties that both countries are currently unable to sever - which I think is also a reason that China is still seen as a "competitor" and not a full-blown "enemy state" in the US. It's restricted to areas where there is a genuine common interest of course. In this situation, I guess the "other actors" would be terror groups, criminals or just reckless corporations - that aren't aligned with either state. Obviously, such a cooperation wouldn't keep China or the US from developing models with those capabilities for their own armies. -- There are lots of other takes with questionable logic in the essay though, such as that China is unable to train frontier models by themselves due to lack of hardware - unless they obtain the training data directly from American frontier models via distillation. Or the assumption that open weights models will be completely opaque and immutable after their release, so a model that passed all the "safety" tests can never be turned back into an "unsafe" model. This seems pretty ridiculous when people are already finetuning open-weight models every day to add new abilities or remove restrictions. And of course that China must not have those abilities because it's an Authoritarian Regime, but Trump USA is totally fine...

xg15 2mo ago
Of course what is Schrödinger's China is the idea that an otherwise adversarial China would simply accept some international ruleset decreed by the US and enforce it on their own territory, without demanding anything in return. E.g. they might say something like "We absolutely agree that international regulation of AI is needed and we're willing to co-sponser an initiative with the US. One of the most pressing matters we see is the spread of despicable misinformation through unregulated models regarding certain events in China's history, the treatment of certain minorities or the status of certain provinces..."

foo42 2mo ago

I agree, although I think the real goal the model providers are going for (both commercial _and_ open weight) is probably power. Just think of the control available to the organisation training the models politicians, business leaders, and citizens are increasingly delegating their thinking to.

jorisw 2mo ago

> The reality is And you know this how?

puszczyk 2mo ago

I don’t disagree with your point on Dario’s conflict of interest. I def think the models are expensive. But why call them overpriced? Compared to what? Even if we take the margin reports at face value, we don’t know their training costs, etc. Curious if this was more of an emotional take or if there’s actual evidence behind it.

topranks 2mo ago
Yeah I don’t think they are over-priced, last I checked all those companies are still losing money hand over fist. I guess the question is more he doesn’t want people catching up by doing cheaper training (through distillation or otherwise), and he definitely doesn’t want companies to spend their money training these models and then _giving them away_, which fundamentally undercuts their commercial model and any way to claw back their investment
gbalduzzi 2mo ago
> But why call them overpriced? Compared to what? Compared to some available Chinese model I guess. For most common tasks the additional intelligence is marginal and the cost is around an order of magnitude higher.
qwery 2mo ago
Compared to how much they want to pay. This is always what 'overpriced' means, as far as I know. The seller's costs/profits/margins aren't a concern for the buyer.

radu_floricica 2mo ago

I was reading the post and thinking "wow, that's pretty clear for a smart man used to writing for other smart men. it'll be really difficult to misunderstand". I read the first couple of comments and stand corrected. A much simpler summary: - open models good. - smart models _can_ be bad - smart open models that can do biotech work are dangerous. worth the hassle of certification _if_ we can get everybody on board with minimalist certification. - banning open models just in US is neither good or bad: is stupid.

npodbielski 2mo ago
Yeah makes everyone agree on banning Chinese models and then force them to use mine models. How I could misunderstand *that*.
hoppp 2mo ago
The biotech work is not even these LLM models. Alphafold and others solving the protein folding problem are revolutionary. An LLM can max provide an instructive tutorial protocol for lab work, but designing novel proteins is not it's job. I am interested to enter the bioinformatics space eventually and the regulation happens at the DNA printer level. Anyone can design yeast nowadays that excretes heroin, but the DNA needed for the genetic modification won't be printed by anyone. As long as DNA printing tech is extremely regulated it's all fine, if home printing becomes a thing then on the other hand maybe people won't even need AI to print deadly pathogens at all. A database lookup will do.

PunchyHamster 2mo ago

Yup, all that talk about "safety" seems to branch into 2 meanings "Dangerous to our bottom line" "We call it dangerous to hype up its abilities"

mcfedr 2mo ago

is it so hard to fathom that China whilst wanting to beat the US in many ways doesn't want a biological war to destroy the world?

cable3 2mo ago

China should cooperate with the United States first for the benefit of its own people. Why should those who do not contribute benefit from it?

bayindirh 2mo ago

In short: "We don't propose a ban to open-weight models. we want to stop these models being developed altogether. If there are no models, there'll be nothing to ban". Oh also: "They ste^H^H^H distill what we have sto^H^H^H used fairly from the world. This is unfair". Lastly: "What if they use their models in their military and local police services like we do? Communism!" As always: https://pbs.twimg.com/media/B_AiI9_XIAA67_t.jpg?name=orig https://pbs.twimg.com/media/B_AiI9_XIAA67_t.jpg?name=orig

Akronymus 2mo ago
A bit off-topic: I occasionally see that triple ^H on HN posts: what does that actually indicate?
- bayindirh 2mo ago
  In the old hardware terminals, and current terminal emulators, ^H (or CTRL+h) is "Backspace signal". It has the effect of pressing backspace. So, ^H^H^H means "delete three characters, excluding '^H's". Like the person typing the comment changes their mind and deletes the characters (or the word) before writing else. It's an stylized way of euphemism. i.e.: Actually I want to say this, but I substitute it for that.
- grahamlee 2mo ago
  Ctrl-H is the rubout/backspace command on old terminals. So if I say it's nuts^H^H^H^Hunwise to ignore UNIX history, I've erased the first word and replaced it with the second.
- TACIXAT 2mo ago
  A caret with a character subtracts 0x40 from its hex value allowing you to insert non-printable characters. Uppercase H is 0x48. That gives us hex 0x08 which is a backspace. https://commons.wikimedia.org/wiki/File:USASCII_code_chart.svg https://commons.wikimedia.org/wiki/File:USASCII_code_chart.s...
- lynguist 2mo ago
  Strikethrough

janpeuker 2mo ago

Regular distilled models show capacity gaps and overfitting that open-weight models don't anymore I think. This focus on distillation as "more compute-efficient" (i.e. cheaper) seems to rather be an excuse for bad (or bubble) investment, fixed hardware dependency and lack of interest in research of efficient compute. Which also shows as climate and sustainability impact. Nvidia signed the open-weight model letter and Europe doesn't have better models either, so chips don't seem like the issue either. I guess good old performance optimisation is just not _cool_ anymore. So they use the same argument as politicians arguing "cheap products" are why tariffs are needed; when instead it's mismanagement. Another HN user wrote the other day "live by the sword, die by the sword".

dalemhurley 2mo ago

Let’s be honest, you don’t need a 3T model for bad actors, in fact you would be better off training a smaller focused model if you want an evil GPT.

ozgung 2mo ago
Good point. I remember the discussions when 3D printers first appeared. People were thinking “bad actors” (“terrorists” then) would use them to print weapons. People love to speculate and exaggerate. But they generally have poor judgements and rarely predict the future correctly.

fnord123 2mo ago

> Schrödinger's China at once is an evil entity looking to use AI for their own nefarious purposes yet also willing to cooperate with their main competitor to prevent other actors (who??) from achieving similar goals (all while under a chip embargo too!!) Very good point. However, if one reads the transcript of the speech that Xi Jinping gave to the World AI Conference on 17 July, we see that he he is very much in favour of AI safety. https://english.www.gov.cn/news/202607/17/content_WS6a5a1172c6d00ca5f9a0c46b.html https://english.www.gov.cn/news/202607/17/content_WS6a5a1172... > Second, we should strengthen risk-awareness and ensure that AI is secure and controllable. AI should be a trusted tool for humanity. We should take seriously the various types of inherent and secondary risks that AI may trigger. We should put in place laws and regulations, technological monitoring, early warning and emergency response systems in order to strengthen the line of security, prevent abuses and malicious use, and ensure that AI is always under human control. In the meantime, we should jointly oppose overstretching the national security concept in the field of AI and placing one country's security over that of others. Now, let's contrast another important part of safety here. Amodei puts the fact that this will need to be a global effort as a mere note that sure, China will need to help too: > Note that to be effective, testing would need to be global, which means even the CCP would need to be on board. I think this may actually be possible: as I wrote in The Adolescence of Technology, limited cooperation around preventing AI biological weapons may be possible because it is in China’s interest too. International collaboration is the focus of Jinping's speech. But one imagines "cooperation" Amodei has in mind if "do what I say" while Jinping has more collaboration in mind here. (Even if you think 'china bad' they deserve credit for collaboration for their open weight models).

fakedang 2mo ago
China has yet to demonstrate weaponization of AI. Meanwhile Anthropic was openly part of the Iran war. I wouldn't give much weight to Amodei's words after that regarding controlling AI weaponization.

irenaeus 2mo ago

I must have missed the part of the letter where China's cooperation was necessary.

VikRubenfeld 2mo ago

On the one hand, Darius argues: > ...banning the use of these models by US businesses does nothing to address this risk, because bad actors are unlikely to be legitimate US businesses. It would protect US AI companies from competition, but that has never been my goal. But on the other, he argues: > All sufficiently capable models, open and closed, should go through mandatory safety testing. Models that don't pass safety testing would be banned. Darius does not appear to be against banning models. He wants the government to have a regulatory body that has the ability to ban models. Then Anthropic can do regulatory capture of that agency and control what models are permitted to be released. Also, during this mandatory safety testing, models would be blocked from use, and by the time the testing was done (probably years) the models would be obsolete.

spiritodim 2mo ago

[flagged]

ibic 2mo ago

"privileged access to chips", nailed it.

gclawes 2mo ago

I think the worse possibility his he actually believes his safety bullshit, believes that he and people who think like him are the only ones with the special knowledge required to do safety correctly, and that they're justified in accumulating total power of this market to people who think like them (which just so happens to be Anthropic).

catigula 2mo ago

This notion that Dario cares only about the bottom line is simply misinformed, in the most charitable interpretation. It doesn't track at all with any of his prior stated beliefs or past actions. It's an absurd claim. It's a baseless conspiracy theory, smuggling in traditional conspiracy mechanics for plausibility.

nedruod 2mo ago

This take is poorly informed and poorly thought out. It's just a cynical vibe, not a cogent argument. If they wanted to squash the competitors, they'd advocate for a ban on Open-Weight models. How else are they going to prevent their release? Anthropic's position is not that, so the cynical nefarious purpose argument isn't even rhetorically true. What they say openly, that open-weight models are harder to secure is absolutely true, and anyone unwilling to acknowledge that and let it inform their own reasoning is a risk. If open-weight models keep being released near the frontier (or even worse, at the frontier), something bad will happen, and we'll have few tools to keep it from getting worse before it gets better. How bad is hard to lay a prediction on, it's perhaps lucky if it's something superficially bad that is a wake up call. You cannot un-release an open-weight model, nor even post-release add a new restriction. Whatever mistakes you made are done and the only option is to fail-forward with every ounce of pain that entails. Diverting from this important point by vibing cynicism is irresponsible. Cynicism is only a powerful tool when it opens you up to deeper reasoning, not when it diverts you from it.

gruturo 2mo ago
> Anthropic's position is not that, so the cynical nefarious purpose argument isn't even rhetorically true Anthropic's position is precisely that, they're just not honest and transparent about it so they are trying to sabotage or delay open weight models in any way they can, direct or otherwise. > If open-weight models keep being released near the frontier (or even worse, at the frontier), something bad will happen, and we'll have few tools to keep it from getting worse before it gets better. How bad is hard to lay a prediction on, it's perhaps lucky if it's something superficially bad that is a wake up call. Calling bullshit. Current open weight models are already pretty close to the frontier - if not there - and the world still spins. Even small(er, relatively) models occasionally surprise everyone by being amazing at certain tasks. Furthermore, closed weight models can do plenty of damage which even their creators occasionally can't control - cue the Huggingface incident of last week and how they had to use GLM to defend themselves. Closed weight models will create a 2-class (or more) world in which some actors will have access others won't (see Mythos, or GPT5.6-Sol when a user has the right flags) and potentially carry out attacks against which the have-nots are... not supposed to be able to defend themselves? Fuck that.

dangoljames 2mo ago

Its a losing battle because anthropic can never be profitable (ditto openAI) They need all the protection they can get and it wont be enough, no matter their market cap.