8 ms·

If you think you need to spend $100B, does using a third-party cloud provider still make sense? It doesn’t matter what sweet deal Amazon is pitching—in that sce

• Tepix 5mo ago
  Sure: If you can't get enough compute by ordering it yourself, make deals with anyone who promises to get you more compute.
• Zababa 5mo ago
  I think it could make sense to not want to own the stack if you think it's going to cost you velocity/focus? Which is probably the play here. But I'm not certain at all.
• loveparade 5mo ago
  Good lucking getting GPUs.
• credit_guy 5mo ago
  Here’s the answer to your queation (from the article) > The Anthropic deal specifically covers Trainium2 through Trainium4 chips, even though Trainium4 chips are not currently available. The latest chip, Trainium3, was released in December. On top of that, Anthropic has secured the option to buy capacity on future Amazon chips as they become available.
  • deskamess 5mo ago
    So it comes down to how much of that $100 bn is in the 'option', I guess. Then it's not an expense at all.
  • superkuh 5mo ago
    Ah. So it's a scalper situation where an unethetical entity buys up all the supply and then resells it for a greater price.
    • t0mas88 5mo ago
      Amazon isn't buying and reselling Trainium chips, those are their in house developed custom chips.
• LogicFailsMe 5mo ago
  Classic time value of money situation. They get access to the HW now so they can continue to grow the business. Of course, if you think AI is just pets.com redux, I can see how you'd think it's already peaked. All those years of very important people insisting Bezos couldn't just pull a switch on reinvesting all the revenue into growing Amazon and then he did exactly that comes to mind.
• Culonavirus 5mo ago
  Only Google and xAI build their own, no? I don't think it's that easy to vertically integrate massive datacenters into a software company. Both Google and xAI (Tesla, SpaceX) have a massive wealth of experience when it comes to building factories.
  • jeffbee 5mo ago
    New level of glazing Elon Musk unlocked. xAI has a vertical integration advantage because Tesla once moved into an old Toyota factory and because once they paid Panasonic to put a Tesla sign outside a Panasonic battery factory. Incredible content.
    • petesergeant 5mo ago
      I would struggle to dislike Elon more, but this seems like you’re some kind of weird anti-Musk fanatic
  • tren_hard 5mo ago
    Facebook and Oracle also build their own, at least before the last couple years where they’ve financed out to new bag holders.
• mitchell_h 5mo ago
  I watched some explain how deepseak got good and the Chinese approach to LLM training. Really wish I could remember it. The premise was China thinks of LLMs not as a thing separate from hardware, but gains efficiencies at each layer of the stack. From Chips to software, it's all integrated and purpose built for training. Wonder if Anthropic is making a mistake by focusing on "consumer" hardware, and not going super specialized.
  • elefanten 5mo ago
    DeepSeek uses merchant silicon like everyone else. edit: I misunderstood, I thought you were implying they designed their own GPUs. nevermind
  • deleted 5mo ago
    [deleted]
  • renewiltord 5mo ago
    It’s fake news predicated on China not being able to get GPUs. But it turns out everyone was getting them their GPUs by serial number swaps in warehouse.
  • jubilanti 5mo ago
    So you watched some random video from some random YouTuber, didn't even remember who made it, so much so you didn't even remember that deepseek isn't spelled "deapseak", didn't bother to even find it or verify, and then you go asserting your memory as fact on a serious discussion forum. Comments like yours add nothing to the discussion.
    • throwa356262 5mo ago
      I belive he does have a valid point. You can throw money and hardware at a problem, but then someone may come along with a great idea and leapfrog you. Just consider that all major AI providers now use deepseeks ideas for efficient training from that first paper.
    • 1738384848 5mo ago
• dktp 5mo ago
  I think these pledges offload some of the risk onto Amazon/Oracle/etc If Anthropic/OpenAI miss projections, infra providers can somewhat likely still turn around and sell it to the next guy or use it themselves. If they have more demand than expected (as Anthropic currently does), vcs will throw money at them and they can outbid the competition If they built it themselves and missed projections it's a much more expensive mistake It's just risk sharing. Infra providers take some of the risk and some of the upside
  • throwup238 5mo ago
    > If they built it themselves and missed projections it's a much more expensive mistake Not if their pricing comes with multiyear commitments for reserved pricing. No doubt they get a huge volume discount but the advertised AWS reserved pricing is already enough for pay for a whole 8x HX00 pod plus the NVIDIA enterprise license plus the staff to manage it after only a one year commitment. On-demand pricing is significantly more expensive so they’re going to be boxed in by errors in capacity planning anyway (as has been happening the last few months). The economics here are absurd unless you’re involved in a giant circular investment scheme to pump up valuations.
    • dweekly 5mo ago
      The pricing models that are published on AWS' website almost certainly have almost nothing to do with the pricing models that are discussed behind closed doors for a $100 billion commitment.
      • deleted 5mo ago
        [deleted]
      • throwup238 5mo ago
        Of course not, but unless they’re getting the sweet heart deal of a lifetime from Amazon of all places, it’s still a hogwash. We’re talking about enough capital to build their own fab and a dozen datacenters*. This deal isn’t going to be buying existing capacity because that’s already stretched, it will be paying for new buildouts. Afterwards Amazon will be milking the machines these commitments buy for nearly a decade. That tradeoff makes sense at a small scale (even up to $X00 million or even billions), but at $Y0 or $Z00 billion? Color me skeptical. There are plenty of other side benefits like upgrading to the newest GPUs every few years, but again we’re talking about paying for new buildouts with upfront commitments anyway. * obviously the timelines, scientific risk, and opportunity cost make this completely infeasible but that’s the scale we’re talking about. It’s a major industrial project on the scale of the thirty year space shuttle program (~$200 billion).
• bombcar 5mo ago
  If you’re sure it’s going to go gangbusters you want to get it all in-house asap. If you’re not sure it’s going to blow the socks off, foisting capital investment on partners is a great deal. See the difference in companies/franchises that always own the land/building and those that always lease.
• bilekas 5mo ago
  I imagine it comes down to if they want to buy hardware every generation, that gets very expensive and depreciates quickly. You've then got a whole load of assets on your books that are technically obsolete for the bleeding edge. This way, AWS buys and maintains the hardware and OpenAI doesn't need to claim it as depreciation ? Just a guess.
• vasco 5mo ago
  That is a project you can work on at any point in the future and the more you delay it the more certain your investment will be about what you really need. But those additions to the PnL are capped to the costs. In the meantime if you work on revenue generating work, that side of PnL is uncapped. So you can either put some engineers on reducing your costs at most by 100% or, if they worked on product ideas they could be working on things that generate over 9000% more revenue.
• lubujackson 5mo ago
  Look at GPU and RAM prices and data center rollout. We have quickly reached Earth's capacity for compute - it is a lot like the housing market. Once there is global saturation, the price to buy becomes increasingly high EVERYWHERE. Let's also not forget that Anthropic moves the market with their purchases and usage. They might literally be unable to buy capacity they need (or project to) and are doing this deal to pave a roadmap for the near-term and to keep global prices (somewhat) down.
  • JumpCrisscross 5mo ago
    > We have quickly reached Earth's capacity for compute Why this versus us being in a temporary bottleneck? Like, railroads became expensive to build everywhere in the 19th century not because we reached Earth's capacity for railroads or whatever, but because we were still tooling up the industry needed to produce them at higher scales.
• samdixon 5mo ago
  From my understanding, if you want to use native Claude in AWS Bedrock, it runs from an AWS datacenter. I'm guessing that's why regardless of running your own stack... they still need a footprint in all the major clouds.
• MeetingsBrowser 5mo ago
  Going from a company with no experience building and operating datacenters to a company with 100B worth of compute is a multi-decade high risk goal.
  • MrBuddyCasino 5mo ago
    xAI built a datacenter in a few weeks, if I remember correctly.
    • 0xbadcafebee 5mo ago
      And they used illegal power to do it (which will now give local poor people health disorders at 4x the national average). They likely violated every law possible in the process, like OSHA standards, overtime. Musk loves to overwork people.
    • MeetingsBrowser 5mo ago
      xAI built the Colossus data center in 122 days (just the physical construction time). Colossus initially had ~200k GPUs. 100B buys you ~1 million high end GPUs running 24/7 for a year at AWS retail prices.
      • Aurornis 5mo ago
        Initial Colossus buildout was 100K GPUs They also reused an existing building that happened to be in the right place at the right time. The larger data center buildouts would almost always need new, dedicated construction.
    • Aurornis 5mo ago
      That’s PR hype. They built it quickly, but they didn’t go from deciding they wanted a data center to having it running in weeks. You can’t even get the hardware at that scale without months or years of order lead time. NVidia doesn’t have warehouses full of compute hardware waiting for someone to come get it. They also reused an existing building. Basically, they put 100,000 GPUs into a building and attached the necessary infrastructure in about half a year. Impressive, but it’s not the same as a $10B/year data center usage commitment like this deal.
• JumpCrisscross 5mo ago
  > It doesn’t matter what sweet deal Amazon is pitching Isn't that almost all that matters when comparing doing something yourself versus paying someone else, in this case Amazon, to do it for you?
• etempleton 5mo ago
  In a rationale business yes, but when everything is basically some form of growth signal to investors to extract even more money from them before the music stops it doesn’t matter.
• avereveard 5mo ago
  Cannot get Tranium anywhere else and NVIDIA commands a super high premium.
  • verdverm 5mo ago
    Similar for Google and their TPU, which Anthropic announced two weeks ago https://www.anthropic.com/news/google-broadcom-partnership-compute https://www.anthropic.com/news/google-broadcom-partnership-c...
• nashashmi 5mo ago
  No. I am guessing that this is only a commitment and they will waver on committing. However there are certain advantages like supply chain that only established companies would have access to. This is also a commitment to spend upto 100B on internal approach and research. I would expect them to come up with their own cpu chip and device design. This will shift the focus to an internal approach. And might make amazon give better prices later down the line
• dgellow 5mo ago
  Anthropic also has their own servers
• DANmode 5mo ago
  > you’d want to own your stack. Everybody does right now, right? But: is it your core competency? Can your firm afford the distraction?
• nickorlow 5mo ago
  AWS exists and has compute right now, spinning up their own HW would take months (at least). This gets them moving quicker.
• 0xbadcafebee 5mo ago
  There is no money or time left to build a $100B stack. All private capital is tapped and banks know it's too risky. They have no choice but to rent.
• IMTDb 5mo ago
  The problem is that at that scale, the alternative is building your own data centers. You'd probably want at least 2 in the US, 2 in Europe, 2 in Asia, maybe 1 in Africa and 1 in LATAM. So 8-10, and you need at least half of them ready "on time." What does "on time" mean? You'll need to negotiate with local authorities, some friendly, some not. Data centers aren't exactly popular neighbors these days. Then negotiate with the local power utility. Fingers crossed the political landscape doesn't shift and your CEO doesn't sign a contract with an army using your product to pick bombing targets, because you'll watch those permits evaporate fast. Then there's sourcing: CPUs, GPUs, memory, networking. You need all of it. Did you know the lead time for an industrial power transformer is 5+ years? Don't get me started on the water treatment pumps and filters you can't even get permitted without. What will you do in the meantime ? You surely aren't gonna get preferential treatment from AWS / Google / ... if they know you are moving away anyway. Your competition will. The risk and complexity are just too big. AI/LLM is already an incredibly complex and brittle environment with huge competition. Getting distracted building data centers isn't enticing for these companies, it's a death sentence.
  • amluto 5mo ago
    Other than data sovereignty, does the data center location really matter that much? Current inference systems are not exactly low latency.
    • Aurornis 5mo ago
      It’s the power and water needs. Large data centers consume as much power as a small city. The location decision is about being able to connect to a power grid that is ready to supply that. Evaporative cooling also needs steady water supply. There are data centers which don’t operate on evaporative cooling but it’s more equipment intensive and expensive. Latency doesn’t matter. You can get fast enough internet connected to these sites much more easily than finding power.
    • sophacles 5mo ago
      * not every task is waiting on the inference. lowering latency on other, serial tasks, can still have a noticable effect. Login, mcp queries, etc. * data transit across the world can be very slow when there's network issues (a fiber is cut somewhere, congestion, bgp does it's thing, etc). having something more local can mitigate this * several countries right now have demented leaders with idiotic cult-like followers. Best not to put all your eggs in those baskets. * wars, earthquakes, fires, floods, and severe weather rarely affect the whole planet at once, but can have rippling effects across a continent. And frankly, the real question isn't "why spread out the DCs?", its "what reason is there to put them close to each other?".
• neya 5mo ago
  I remember seeing this extremely shocking graph of top AI companies on Facebook on how the money just keeps changing hands between a handful of companies. Almost seemed like a scam.
  • Aurornis 5mo ago
    Money doesn’t just flow around with nothing exchanged. The money is in payment for goods and services. It’s common even for smaller companies to do mutually beneficial business with each other. It’s actually helpful to do business with people who are also your customers because you have a relationship with them and you also have leverage: They are extra incentivized to treat you well because they don’t want to upset any of the other business you have with them.
  • neffy 5mo ago
    It is a similar kind of lending loop to that which went on during the late 1990's leading up to the 2000 crash. A lends to B lends to C lends to A. There is a famous quote from the polish economist Kalecki, that "economics is the science of mistaking a stock for a flow". Essentially this form of lending continues while everybody can make interest payments, and blows up horribly as soon as somebody can´t - as I have no doubt all those concerned are fully aware.
    • neya 5mo ago
      > It is a similar kind of lending loop to that which went on during the late 1990's leading up to the 2000 crash. Interesting...
  • bsder 5mo ago
    It's the Carly Fiorina playbook. Welcome back to the TeleBomb! Lucent sends their regards.
• jimjeffers 5mo ago
  My guess is they are bound not by capital as much as they are physical resources. Amazon probably has the land, crews, etc. to build out more data centers faster than Anthropic can right now. The scarce resources are the chips and electricians not the money!
• dec0dedab0de 5mo ago
  They're not trying to build a sustainable business. They're trying to get as much market share and lock-in as possible before the bubble bursts. This makes a ton of sense from that perspective. It probably would be cheaper for them in the long run to own their own hardware, but they are paying AWS for their expertise so they can focus on what they do. If it doesn't work out, it also sets them up for a merger with Amazon. I do think a ton of businesses would benefit from running their own hardware, but they're not getting five billion dollars to stay on the cloud.
• tahoeskibum 5mo ago
  That is why only SpaceX/X.ai has the true advantage...
  • hnav 5mo ago
    maybe in the game of promising ludicrous things. There's no realistic plan to put compute in space.