4 ms·
* Frontier models need infinite high quality private IP to keep them fed. Forcing an IP theft funnel ensures big lab survival and model intelligence growth. *
by nullbio 23d ago
* Frontier models need infinite high quality private IP to keep them fed. Forcing an IP theft funnel ensures big lab survival and model intelligence growth.
* Open-weight models are 1month behind frontier models. Cheaper, faster, private (no IP theft), steerable (you can security harden your own software without safeguard triggers). No sane business would keep using these API services if they didn't have to. The labs stand to lose a fortune.
* Dario has stacked the deck at METR, who are funded by all the same NGOs who are funded by Anthropic and its investors. METR is full of ex-Anthropic employees with massive equity stakes. If they manage to position METR as the "independent evaluator" for the industry, they control what gets evaluated, how, and who passes.
* Creating a gap between what the public knows exists (model capabilities) and what is used in secret allows it to be weaponized against other nations and the public.
* No requirement for public disclosure on model capabilities allows them to feign they've hit intelligence ceilings while they secretly RSI to the moon with better and better chips.
* Slowly but surely, this will allow the big labs to swallow the entire economy and every single business on Earth, by cloning and automating.
This, and many more reasons.
The labs need to feel more pressure to be held accountable for the incidents they cause (HF incident, etc), so they have an incentive to ensure it does not happen again.
- Turn_Trout 22d agoDario's post [1] commits to direct evaluators that can, among other abilities, expose secret RSI. He wants that made law. Do you have a source on METR employees retaining massive equity stakes? [1] https://darioamodei.com/post/we-must-pace-the-frontier https://darioamodei.com/post/we-must-pace-the-frontier
- nullbio 22d agoJoe Benton left Anthropic a day before Dario's post, to work for METR evaluations. He was with Anthropic for over a year. He did the same thing that Jacob did (big song and dance about AI apocalypse, media interviews all over the place). He managed the Scalable Oversight team at Anthropic and was the research lead for the Anthropic Fellows Program. So he has equity, and likely lots of it. Then you have Josh Engels quitting DeepMind to work for METR the day before as well, doing the exact same thing. Again, doomer drama all over socials, interviews, and so on. Did I mention METR is founded by an ex-OpenAI researcher? Now you have Demis Hassabis, Sam Altman and Dario, all circlejerking eachother on X saying "we all agree with Dario" - while they ask to be "regulated" by the company that has all of their combined equity-holding ex-employees in it. METR's salaries are listing around 500k/yr. Gee, I wonder where this non-profit with ~35 people is getting all of its money? So the fact that Dario tries to frame it as an "independent third party" is all the evidence you need to know that Dario is a pathological liar and always will be. --- Some more info: Dario's sister, president of Anthropic, is married to the co-founder of Open Philanthropy. The two largest AI doomer NGOs, Center for AI Safety (CAIS) and the Future of Life Institute (FLI), have both received many millions of dollars from them. Ajeya Cotra worked at Open Philanthropy/Coefficient Giving for roughly nine years, including leading its technical AI-safety program in 2024 and contributing to AI-giving strategy in 2025. She subsequently left Coefficient and joined METR, where she is now technical staff. Ajeya is married to Paul Christiano, who founded Alignment Research Center (ARC). Alignment Research Center donated ~$4.5mil to METR. Good Ventures is a funding partner of Open Philanthropy, who funded Jacob Coxon (the first of the Anthropic employees going viral in the media) via a scholarship.
- Turn_Trout 22d ago> Good Ventures is a funding partner of Open Philanthropy, who funded Jacob Coxon (the first of the Anthropic employees going viral in the media) via a scholarship. This conspiracy theory is truly crazy. A $20K scholarship in 2022 is supposed to explain Coxon walking away from unvested equity for a company worth over $950 billion dollars?
- nullbio 22d agoObviously not, and that's not what it demonstrates. It demonstrates relationships, collusion and favoritism. I don't think Coxon was ever planning on or entitled to taking equity, I think this was the plan from the beginning and why he was hired for 6 weeks to begin with.
- nativeit 22d agoThe company is valued over $950B. It remains to be seen how much it’s worth.
- ben_w 22d agoI know people at METR and Anthropic. When they tell you they're worried the tech they're working on may kill everyone despite their best efforts, perhaps believe them.
- calgoo 22d agoOr you know, stop working on it if its that dangerous? This whole thing of a bunch of employees saying that they are scared of building what they are building, but do it anyway because they are somehow going to make it different? Their model has been used in the planning of mass murdering in war as well as spying on the entire worlds population as well as helping ICE out in the US. They need to stop this BS fearmongering or actually stand up and do something about it. A government regulation is not the answer, especially when its done in a country that is run by a want to be dictator.
- int_19h 22d agoAnthropic specifically is basically saying that they believe it's even more dangerous if someone else gets to AGI before they do, so they have to either stop everyone or not stop themselves. I personally disagree with that take - and, as you note, it's hard to take seriously ethical wrangles from a company that literally sued the government in court to allow their models to be used by Palantir of all people. But if one genuinely believes that it's the robots themselves (rather than the people controlling the robots) that will kill us all, it's not inconsistent.
- the_optimist 22d agoIt’s a bit of a self-serving argument, don’t you think? And how does that relate to the ask for oligopoly licensing within global democracy? “We must build the nuclear bomb first in order to make sure no one else builds one.” This the most nonsense, disingenuous argument imaginable.
- forgotone 22d agoThe people at these companies are dumb as rocks. All “tech workers” are.
- user43928 22d agoOpen-weight models are not one month behind. In fact they still have not caught up with February's Mythos, indicating they are more than half a year behind.
- cgio 22d agoFor practical uses they are there. Arguably the frontier models are worse for some of these practical tasks. And keep in mind, people will use maybe frontier for 1/10th of the work, planning and review, and go open source for rest. The question is if they manage to impose outside us. If not, they are losing competitiveness.
- user43928 22d agoI always thought switching from a SOTA model to a dumber model after planning was a terrible idea. Mostly I heard this from people who I got the impression have little experience in developing greenfield software with agentic AI. Often the same people who talk about spec frameworks. I fundamentally disagree with the approach. I believe the ability to autonomously evaluate, test, and adjust during long horizon tasks is critical to using AI efficiently.
- rescbr 22d agoWell, it's the enterprise software house pipeline... The software architect writes the spec, hands it down to the implementation team, senior leads, junior devs or offshore teams codes it. I also disagree with the approach, this is cargo-culting the existing ways of working.
- jurgenburgen 22d ago> this is cargo-culting the existing ways of working. I think you mean the existing anti-patterns. They don’t call them ivory tower architects for nothing.
- lukewarm707 22d ago
- aatd86 22d agoThat is what the Palantir guy (Karp) has been warning against. People/business need to keep their IP instead of throwing it all into Claude and whatnot. Risk being the worst aspects of communism which I think he meant centralization of decision, asymmetric supply/demand for compute (they lock you in), and the tech overloard Anthropic/OpenAI/xAI being in competition with everyone;s business all of a sudden with much more data. An unfair advantage in markets made super competitive all of a sudden. A winner takes all attempt. This is not sustainable anyway, the scale at which they want to control data flows. Time was money, now data is money and they are too greedy for it. All this agitation is just a silly attempt at constraining competition. The danger is not the AI, it is having all your systems connected. Overreliance on networked tech.
- bbor 22d agoOk but the first point is just not based in reality whatsoever, sorry.
- Bluestein 22d ago> Open-weight models are 1month behind frontier models If correct, grip is tightening. Used to be 1y, then 6 months ...
- enknee1 22d ago- Frontier models need infinite high quality private IP to keep them fed. Forcing an IP theft funnel ensures big lab survival and model intelligence growth. While the data pipeline is necessary, the assumed source for data (in this case people) is incorrect. Right now models are "aligned" because of the RL-based people pipeline. But there's a whole world of readily available data that does not require a human to access. Put sensors on a vacuum, install lidar on the front of a car, sell phones with cameras on them, and you amass data for the creation of models of the world that don't include the need for a human filter. Richard Sutton and many others see this approach as the only viable path to AGI. Such models would be truly alien and unaccessibly dangerous. We're probably building them right now.
- deleted 22d ago[deleted]
- datadrivenangel 22d agoOpen weight models are all built on IP theft as well. Also more like 6-12 months behind, maybe 18 but it's hard to say.