5 ms·
I have never understood this line of thinking. Why would a company warn that their technology is dangerous if it wasn't? Looking at the ongoing Mythos export co
by ajyoon 3mo ago
I have never understood this line of thinking. Why would a company warn that their technology is dangerous if it wasn't? Looking at the ongoing Mythos export control issue, it's obviously not good for profits. If you assume AGI is possible, you must agree that it would be at the very least profoundly destabilizing. The companies are built around this assumption.
- prasadjoglekar 3mo agoSetting up for the eventual bailout. Remember - "we can't let banks fail or it'll be a depression"?
- SpicyLemonZest 3mo agoI remember a meme that spread to that effect, but it was similarly divorced from reality. Hundreds of banks failed during the financial crisis, including multiple which were very large. There were a few cases like Wachovia, where an unhealthy bank was required to sell itself in order to avoid a technical failure that would have impacted over 10 million Americans' access to their money. But this was still unpleasant for existing Wachovia shareholders.
- watwut 3mo agoI remember large banks that caused the failure to be bailed out. And talk of trickle down economics, except that average joe was not bailed outed, but punished. I remember robo signing of home foreclosures with no checks and people loosing houses despite paying debts.
- SpicyLemonZest 3mo ago> I remember large banks that caused the failure to be bailed out. And talk of trickle down economics, except that average joe was not bailed outed, but punished. Another thing that is commonly remembered but did not really happen as described. If you had to identify any particular bank as having caused the crisis, it would have to be Lehman Brothers, which was not bailed out. But really, it was a systemic crisis to which TARP was deployed as a systemic solution. Much of the "bailout" talk is based on memories of AIG, which is not a bank and behaved far more irresponsibly than any bank did; they seem not to have bothered hedging their exposure to widespread mortgage defaults at all. > I remember robo signing of home foreclosures with no checks and people loosing houses despite paying debts. This did happen, yes, and that was bad.
- _aavaa_ 3mo ago*why would they warn if it they didn’t believe it. 2 words: regulatory capture. Ban Chinese models because they’re “bad”. (can’t have head to head competition). Bad open weight model (can’t let people escape the subscription model).
- ajyoon 3mo agoPeople have been warning about the dangers of advanced AI since long before open weight competition was a consideration.
- _aavaa_ 3mo agoSure, but they’ve either been the kool-aid-drinking p-doomers, or academic discussions about giving them access to weapons. Now we have real companies whose valuation depends on people using their services.
- TremendousJudge 3mo agoyes, and all their abstract, philosophical thought is now being used to justify high valuations by these companies. if what they are making is the new nuclear bomb, well then they must be very valuable indeed.
- ben_w 3mo ago> 2 words: regulatory capture. > Ban Chinese models because they’re “bad”. (can’t have head to head competition). The current scale of current + planned data centres only makes sense if the US exports inference worldwide. 30% of your current total electrical generation capacity, and equivalent to 76% of your total electrical demand in 2022. At least, assuming the "33 GW installed, 300 GW more planned" claim I've heard was not itself an AI hallucination. And at least OpenAI and Anthropic were persuing the same argumemts continuously to their initial founding, before such capture was plausible. I don't only mean that GPT-2 was, famously, when OpenAI surprised the world with (paraphrased) "we think this is a good point to start practicing not releasing the weights, before models get too dangerous" or as a surprising number of people interpret it even when I link to OpenAI's own post "we think this is … too dangerous". I mean even before that. I wouldn't disagree if you say Musk is full of BS, but he did donate to OpenAI when it was created as a nonprofit, and his wealth was 2 orders of magnitude smaller than today. Back then he was talking about risks, "summoning the demon", now he's the one demanding saying "robot army" to justify a bigger compensation package from Tesla.
- chongli 3mo agoWhy would a company warn that their technology is dangerous if it wasn't? The whole point of the article is to answer this question, and here's the answer: Because all the AI doom fear-mongering is driving sky-high valuations. The more the public panics, the more investors open their wallets.
- ajyoon 3mo agoIs there any evidence of this? It seems like a thing people say all the time, but completely unsubstantiated.
- chongli 3mo agoWhat else is there? The product as delivered today doesn't come anywhere near a justification for these valuations. All of it is built on an expectation of future capabilities, and that's where the Dario-penned doom papers come in.
- dartharva 3mo agoRegulatory capture and false AGI hyping, what else? You'd have to be willingly ignoring the writings on the wall if you still haven't identified it yet.
- britch 3mo ago> Why would a company warn that their technology is dangerous if it wasn't? Because it makes their technology seem more important and powerful? Do you usually believe 100% what a company says about itself? > Mythos export control issue, it's obviously not good for profits Who cares about profits? None of these companies make money like a business currently. It's all speculative valuation and influenced by the same hype/doom cycle
- SpicyLemonZest 3mo agoWhen people discuss the negative effects of climate change, do you think that boosts fossil fuel company valuations by making their products seem more important and powerful?
- gentooflux 3mo agoHell yes. More than a third of Americans would run their two and a half ton pickup trucks on burning copies of An Inconvenient Truth if it were at all possible
- watwut 3mo agoFossil fuel companies do everything in their power to stop that discussion. However, arms manufacturers are doing everything in their power to make you feel unsafe and weak and scared. That way, you are more likely to buy a gun.
- jmye 3mo ago> it's obviously not good for profits. Weird to think profits matter half as much as ever-increasing valuations, driven by memetic bullshit. E.g. the entire point of the article you're commenting on.
- ajyoon 3mo agoAnthropic's valuation is driven by its revenue growth. Export controls harm this. Hype certainly plays a role, but it's simply not the case that revenue is not a major part of the picture.
- OnlyANeurosci 3mo agoWhat revenue growth? They go from net loss in year 1 to larger net loss in year 2, always have.
- manoDev 3mo agoIt’s the same narrative as “the communists are developing nukes so we need to develop first”.
- bryan0 3mo agoI agree. There are 2 groups of people: 1. Those, like OP, that believe AI danger and disruption is all hype to boost valuations. 2. Those, like myself, that believe AI danger and disruption is very real and presents dilemmas (but also opportunities) for society, so we must tread carefully. The first position is not logically consistent with reality. The danger is real: we have already seen hints about how this will impact everything from jobs to warfare to mental health. that danger does not increase valuation, in fact it does the opposite because of the need for government regulation.
- ekelsen 3mo agoI don't think the first position is steel manned by assuming those holding it actually believe there are no potential problems or ways to use the technology badly. But that belief does not imply the doom rhetoric is necessary or the best way to approach those issues.
- shard972 3mo ago[dead]
- mrandish 3mo ago> in fact it does the opposite because of the need for government regulation. For frontier leaders, regulation is a competitive moat. At trillion dollar scale, compliance teams, legal and lobbyists are a tiny fraction of opex. For emerging startups, early regulation of a new market is a substantial barrier to entry. That said, I think Anthropic blundered into their current Fable mess unintentionally by underestimating just how much they pissed off the current administration by refusing to let the DoD use Ant's AI's for all the domestic spying and lethal combat ops they want.
- JMiao 3mo agothat's an inaccurate characterization on #1. he's not saying ai risk is _all_ hype to boost valuations, but that it's misused to serve purposes other than warning.
- Barrin92 3mo ago>Why would a company warn that their technology is dangerous if it wasn't? Because Americans are obsessed with eschatological narratives and action hero stories. I talked to a Palantir guy once who told me that he loves it when journalists describe his company as a James Bond villain because every single time the market cap goes up. If they said that they're SAP with Call of Duty marketing they'd actually lose money. In other countries this strategy might not check out but in the US it's better to pretend to be a supervillain than to be boring
- ben_w 3mo ago> If they said that they're SAP with Call of Duty marketing they'd actually lose money. In other countries this strategy might not check out but in the US it's better to pretend to be a supervillain than to be boring Well, that explains Musk.
- mrandish 3mo ago> Why would a company warn that their technology is dangerous if it wasn't? It helps to look at the history of the AI Alarmist, EA-adjacent mindset. They believe AI could be Dangerous (<--- capital D) if not responsibly managed by the right people (spoiler: themselves or politicians who listen to them). The capital D means more than just some economic and employment disruption, they're thinking much bigger, like "existential threat to humanity" big. The most extreme alarmists are basically LARPing Terminator 2 and want to shut it down before SkyNet takes over but the more moderate alarmists believe AI is too potentially beneficial to walk away from. And they acknowledge other nations and various bad actors won't stop anyway, so they believe it's their sacred duty to move forward quickly albeit very responsibly. Yes, this leads to some conflicted reasoning and priorities, like they need to keep gaining access to gobs of capital to ensure they are the ones to reach AGI first, as that's the only way to ensure "god" is benevolent. While I'm sure many of the top AI leadership are primarily (or purely) mercenary, I also think some, if not many, other execs and employees sincerely believe (or at least hope) they are "the good guys" playing a crucial role in an era of historical importance. Obviously, this has some heavy vibes as well as a being quite a buff to one's self-importance. They face each new day feeling the weight of bearing such a consequential role in shaping the future of humanity. Us mere mortals can only pray their wisdom and altruistic ethics match their humility. </s> So... yeah, that's why we see some weirdly whiplash messaging.
- ekelsen 3mo agoBecause it makes them feel important because they are developing something so dangerous everyone else ought to listen to what they say. It's about ego, not profits.
- grebc 3mo agoThey banned exporting bad cryptography algorithms/code 30 years ago too. People just love to talk about anything that tickling their fancies. It’s all a fugazi.
- anon373839 3mo ago> Looking at the ongoing Mythos export control issue, it's obviously not good for profits. This is not obvious at all. Anthropic's biggest investor was the catalyst for this action in referring the "jailbreak" (if you can even call it that) to the government. Now, Anthropic is sitting around a table with government officials who are designing benchmarks that will determine what Anthropic's competitors will be allowed to build. At this point, I have no evidence to rule out the possibility that Anthropic's leadership purposefully sought this outcome.
- watwut 3mo agoThey did not wanted export controls, but they did wanted everyone so scared and thus pay for license. The decision backfiring does not imply it was not intended to be self serving. If the company really think their product is dangerous, they should stop making it increasingly dangerous. Antromorphic claims is dangerous and that they need more money to make it even more dangerous. And the fearmongering is beneficial for them so far.
- siren2026 3mo ago>> Looking at the ongoing Mythos export control issue, it's obviously not good for profits We must live in different planets. The whole episode about Mythos has been the absolute best free PR move ever designed. At this point everyone in tech assumes Anthropic and Mythos are the absolute best models. All of this got achieved without spending a single $$ in advertisement. This model is also probably really dangerous, not denying that. But they definitely used it as a huge PR tool.
- ben_w 3mo ago> The whole episode about Mythos has been the absolute best free PR move ever designed. Great PR. For a product nobody can pay for. And a high chance that when anyone can, the market shrinks from "worldwide internet users" to "the USA only". And it's known to be a highly dynamic market: based on the historical average, it only takes 11 weeks to be completely trounced by someone else's model, so if you're out of the market that long you may as well not have bothered.
- siren2026 3mo agoHow many non-tech people knew about Anthropic before? How many know about Anthropic since this episode? Still don't see it? Still don't see how the huge IPO coming up soon could profit from this? Yes, this was a brilliant PR move (independent from the actual benefit of the model).
- ben_w 3mo ago> Still don't see it? Still don't see how the huge IPO coming up soon could profit from this? "Would you like to buy Anthropic shares?" "Anthropic? Didn't the government ban them?" "Yes" "I'll pass"
- tim333 3mo ago>For a product nobody can pay for I remember then Thefacebook.com was only available at certain universities and everyone felt slight envy that they weren't included. Didn't work out too bad for them.
- vintermann 3mo ago> Why would a company warn that their technology is dangerous if it wasn't? That's a bit like asking, "why would someone claim to have a lot of money if they didn't?" It's step 1 in countless scams. You can't think of any ways to exploit people by convincing them you have technology which might rule the world soon?
- xg15 3mo agoYou could also turn this around: Why would a company put up warnings about their product, or at least more than they're legally obligated to. The tobacco and social media industries certainly didn't go "this product has the risk to do great harm, but we also believe in its potential, so we'll go with a responsible approach". The only examples of companies voluntarily putting up warnings I can think of are those obviously tongue-in-cheek warnings "this book/game/series may cause addiction because it's just so good" etc.
- NichoPaolucci 3mo agoTo me, they're selling the "power" of their product by mentioning the danger. "It's TOO powerful to even release yet!" Whether or not that level of power exists, that is definitely how they're pitching it.
- altmanaltman 3mo agoA common line of logic would be that a company would never put a dangerous sticker on their product unless they absolutely have to by regulation. So if they do it, it absolutely means they see a benefit of doing it. Also why do you think the Mythos thing is bad for profits? Anthropic hyped the model up too hard and didn't even have the compute to serve it properly to all users. The export control issue has not affected Anthropic's revenue and only made its top of the line products seem far more impressive than they actually are. People don't think Nvidia is a bad company if the government puts export controls on their products. For Anthropic, valuation and public perception is 100x more important before the IPO than revenue or profitablity.
- cainxinth 3mo agoWhen the U.S. first legalized pharmaceutical ads on TV, one of the requirements was that the drug companies had to disclose side effects during the ads. The companies chafed at that, thinking it would make the drugs less appealing, and only included the bare minimum the rules required them to. But their own data eventually showed that consumers actually responded positively to hearing the side effects. They found that people think if a drug is powerful, it must also have strong side effects. So instead of just scaring them, it reassured them the product was good. After learning that, the drug makers started including more side effects in the ads than they were required to.
- binary132 3mo agothe headline literally explains that it is to maximize hypothetical valuation