4 ms·

Of course they've stood it out. The rate of change and the R&D expenditure is off the charts. It buys them marginal utility to hire AI talent at incredible sala

by bane 1y ago

Of course they've stood it out. The rate of change and the R&D expenditure is off the charts. It buys them marginal utility to hire AI talent at incredible salaries to keep them at table stakes. Meanwhile, the models are getting larger and more complex, with more users, putting the support infrastructure well beyond what individuals and even small companies can afford to outright buy. You can easily spend well over a million on even basic infrastructure to try to support some of the newer models and make it available to a few end users. As a point of strategy for individuals and small entities, it really is cheaper in this case to spin up some AWS instances for a bit to do some LLM work and then spin them down when not in use. So if you were AWS do you mine for gold? Or do you sell shovels?

amy_petrik 1y ago
it's also a "moneyball" situation - hire hyper expensive AI stars or get 10-fold? 100-fold? cheaper smart AI folk sans star power
boplicity 1y ago
This follows my take, in terms of where the profitability will be long term. It will be with the hardware vendors, and not the model creators. Time will see if I'm right, but, as hard as it is to create a good model, it does seem to be something that can be replicated by others.
- mrits 1y ago
  Not exactly going out on a limb here by predicting hardware manufacturers are going to make money on AI
  - boplicity 1y ago
    True, but I'm arguing that it won't be nearly as profitable to be on the software end.
- somenameforme 1y ago
  More cynically, the way to get rich during a gold rush is to sell shovels.
gundmc 1y ago
If AWS instances in this analogy are shovels, what are the GPUs?
- matsz 1y ago
  One layer up, the blade. Silicon would be the iron in this analogy. The physical server itself would be the wooden handle, I guess.
- GarnetFloride 1y ago
  Picks or any of the other required equipment.
- SAI_Peregrinus 1y ago
  Raw iron, timber, & coal supplies.
- bane 1y ago
  GPUs are the land that's staked. They're of limited intrinsic value in the sense that they're in the way of getting to the gold (models), but there's a small supply of it so they go for a lot on the market. But if there was literally any other way to make the models even for a few percent cheaper, they model builders would move to that. On the inference side, most of the cloud providers are looking for pretty much any way to server that up more cheaply, with custom TPUs, or other tensor units of some type. We saw this with crypto mining where truckloads of expensive GPUs were dumped in the trash after the proof of work became so hard it became not worth the cost of electricity to keep on that generation of card.
npalli 1y ago
That whole “sell shovels” thing never really made sense, even in the pre-GPU hyperscaler days. BTW, the shovel is GPU (owned by NVidia for now). AWS, Azure, GCP weren’t just renting servers. They built whole platforms - databases, ML stacks, dev tools, security. Way more than shovels. The moat was owning the stack. MS used Azure to power Office and now Copilot. Google used infra to juice Search, YouTube, Ads. Even Amazon used it for retail + Alexa. They were mining gold and selling shovels. And raw compute was never where the money was. Renting VMs was the cheap layer. The profits came from all the higher level services built on top. Now with AI it’s even more obvious: Models drive the workloads. OpenAI/Anthropic/DeepMind aren’t just customers, they’re shaping the infra itself. Whoever owns the models sets the rules. No models = no moat. If AWS isn’t building frontier models, it’s just reselling Nvidia GPUs while MS + Google wrap their clouds around first party models + SDKs. That pulls customers deeper into their stacks, not Amazon’s. Falling behind compounds. Training/deploying models forces infra breakthroughs (chips, compilers, scaling). If AWS isn’t in that game, they’ll eventually struggle to even run other ppl’s models as well as rivals. So if Amazon “sits this one out,” it’s not just losing bragging rights. It’s giving up control of the future of compute.
- deleted 1y ago
  [deleted]
- bitmasher9 1y ago
  Are modals the future of compute? I’m not 100% convinced this is true. Additionally, I’m not convinced that a waiting pattern right now sets Amazon up for a point of no return. It seems plausible for Amazon to pull an Apple here, to wait until technology is more mature and use their unique position to provide a quality offering.
  - enos_feedler 1y ago
    The problem is Amazon is usually Apples non competitive cloud partner. They can’t both sit this out. AWS needs to learn in a hurry whether they should be in the model business to supply Apple with Siri LLMs. Bc if not Apple is going to Google (and Google cloud). Thats not good for AWS. Amazon is in a bit of a bind bc they should be acquiring Anthropic but not at bubble prices.
doctorpangloss 1y ago
> Blah blah blah, money this, money that See, that’s the problem with what Amazon has done to you. It’s always about money with you guys. Good research is about the opposite of money. The people who don’t know what that means, who can’t fathom to understand what “the opposite of money” means without turning everything into a contrived story about money: they can’t do good R&D. Every single great R&D director will tell you this, and a bunch of people will downvote this comment, who have never been in a meaningful R&D role. A good research culture is capable of listening to broad, generalized, completely accurate criticism in public and not downvote. Downvoting is your problem guys! OpenAI has a million little haters out there and do you know how much time their people spend downvoting comments online? Zero. And honestly they’re paid way better than the poor souls who have wound up at Amazon, so it’s really, truly the case that none of this money money money culture really adds up to much for the little guy. If there’s any one person to point the finger at - like why does Amazon, with its vast resources and tremendous talent, produce basically zero meaningful publicly influential research - it’s Jeff Bezos. You’re talking about strategy? The guy in charge is a colossal piece of shit, with a piece of shit girlfriend and a piece of shit world view, at least as bad as Larry Ellison, whose only redeeming factor is that MacKenzie Scott is a much smarter person than he ever was.

by bane 1y ago

amy_petrik 1y ago

it's also a "moneyball" situation - hire hyper expensive AI stars or get 10-fold? 100-fold? cheaper smart AI folk sans star power

boplicity 1y ago

This follows my take, in terms of where the profitability will be long term. It will be with the hardware vendors, and not the model creators. Time will see if I'm right, but, as hard as it is to create a good model, it does seem to be something that can be replicated by others.

mrits 1y ago
Not exactly going out on a limb here by predicting hardware manufacturers are going to make money on AI
- boplicity 1y ago
  True, but I'm arguing that it won't be nearly as profitable to be on the software end.
somenameforme 1y ago
More cynically, the way to get rich during a gold rush is to sell shovels.

gundmc 1y ago

If AWS instances in this analogy are shovels, what are the GPUs?

matsz 1y ago
One layer up, the blade. Silicon would be the iron in this analogy. The physical server itself would be the wooden handle, I guess.
GarnetFloride 1y ago
Picks or any of the other required equipment.
SAI_Peregrinus 1y ago
Raw iron, timber, & coal supplies.
bane 1y ago
GPUs are the land that's staked. They're of limited intrinsic value in the sense that they're in the way of getting to the gold (models), but there's a small supply of it so they go for a lot on the market. But if there was literally any other way to make the models even for a few percent cheaper, they model builders would move to that. On the inference side, most of the cloud providers are looking for pretty much any way to server that up more cheaply, with custom TPUs, or other tensor units of some type. We saw this with crypto mining where truckloads of expensive GPUs were dumped in the trash after the proof of work became so hard it became not worth the cost of electricity to keep on that generation of card.

npalli 1y ago

That whole “sell shovels” thing never really made sense, even in the pre-GPU hyperscaler days. BTW, the shovel is GPU (owned by NVidia for now). AWS, Azure, GCP weren’t just renting servers. They built whole platforms - databases, ML stacks, dev tools, security. Way more than shovels. The moat was owning the stack. MS used Azure to power Office and now Copilot. Google used infra to juice Search, YouTube, Ads. Even Amazon used it for retail + Alexa. They were mining gold and selling shovels. And raw compute was never where the money was. Renting VMs was the cheap layer. The profits came from all the higher level services built on top. Now with AI it’s even more obvious: Models drive the workloads. OpenAI/Anthropic/DeepMind aren’t just customers, they’re shaping the infra itself. Whoever owns the models sets the rules. No models = no moat. If AWS isn’t building frontier models, it’s just reselling Nvidia GPUs while MS + Google wrap their clouds around first party models + SDKs. That pulls customers deeper into their stacks, not Amazon’s. Falling behind compounds. Training/deploying models forces infra breakthroughs (chips, compilers, scaling). If AWS isn’t in that game, they’ll eventually struggle to even run other ppl’s models as well as rivals. So if Amazon “sits this one out,” it’s not just losing bragging rights. It’s giving up control of the future of compute.

deleted 1y ago
[deleted]
bitmasher9 1y ago
Are modals the future of compute? I’m not 100% convinced this is true. Additionally, I’m not convinced that a waiting pattern right now sets Amazon up for a point of no return. It seems plausible for Amazon to pull an Apple here, to wait until technology is more mature and use their unique position to provide a quality offering.
- enos_feedler 1y ago
  The problem is Amazon is usually Apples non competitive cloud partner. They can’t both sit this out. AWS needs to learn in a hurry whether they should be in the model business to supply Apple with Siri LLMs. Bc if not Apple is going to Google (and Google cloud). Thats not good for AWS. Amazon is in a bit of a bind bc they should be acquiring Anthropic but not at bubble prices.

doctorpangloss 1y ago

> Blah blah blah, money this, money that See, that’s the problem with what Amazon has done to you. It’s always about money with you guys. Good research is about the opposite of money. The people who don’t know what that means, who can’t fathom to understand what “the opposite of money” means without turning everything into a contrived story about money: they can’t do good R&D. Every single great R&D director will tell you this, and a bunch of people will downvote this comment, who have never been in a meaningful R&D role. A good research culture is capable of listening to broad, generalized, completely accurate criticism in public and not downvote. Downvoting is your problem guys! OpenAI has a million little haters out there and do you know how much time their people spend downvoting comments online? Zero. And honestly they’re paid way better than the poor souls who have wound up at Amazon, so it’s really, truly the case that none of this money money money culture really adds up to much for the little guy. If there’s any one person to point the finger at - like why does Amazon, with its vast resources and tremendous talent, produce basically zero meaningful publicly influential research - it’s Jeff Bezos. You’re talking about strategy? The guy in charge is a colossal piece of shit, with a piece of shit girlfriend and a piece of shit world view, at least as bad as Larry Ellison, whose only redeeming factor is that MacKenzie Scott is a much smarter person than he ever was.