4 ms·

> Small models are rapidly growing in capability, require less compute to train and serve According to Jevons' paradox a reduction in resource requirements (im

> Small models are rapidly growing in capability, require less compute to train and serve According to Jevons' paradox a reduction in resource requirements (improved resource efficiency for the same payoff) leads to an increase in demand. This stops working when demand for compute is completely exhausted, but we are very far from that. There's even some very silly predictions floating around (see the latest Dwarkesh Patel podcast) that say compute will soon be most of the economy, even dictating market interest rates. Now, that has to be wrong, but the directional outlook is closer to correct than "very small and efficient models mean there will be ~0 demand for HPC-like compute".

kemiller 2mo ago
Yeah, I think there's a tendency to underestimate how much demand is still gated behind cost constraints. The market for this is HUGE. The PC era, call it 1975-2005, was one of the greatest wealth creation events in history, was characterized by the cost of the underlying commodity dropping mercilessly for the whole time. Each time it did, the space of problem you could solve with a PC would increase, to the point that by the end, they were both replacing mainframes and powering users who do nothing but chat and post cat pictures. Could there be a correction in the short run? Quite possibly. I think an underestimated last mile problem is just the massive weight of bureaucracy and human process inertia. But in the long run, cheap, efficient intelligence is a new engineering capability that we've just begun to even explore.
- GiorgioG 2mo ago
  > The market for this is HUGE. Source(s)?
  - formerly_proven 2mo ago
    The backlog of every software team on the planet being anywhere between 1 and 100 years long at human burn rates.
- minraws 2mo ago
  > how much demand is still gated behind cost constraints. The market for this is HUGE. I think this misses the actual limits here. The problem isn't demand it's, "how much people are willing to spend on it". Cheap AI has to be served on cheap compute, and if inference gets cheap enough to unlock massive usage numbers, by definition it also doesn't require anywhere near as much infrastructure per unit of demand. Take DeepSeek serving ~100T tokens/day, depending on workload and utilization, you're potentially talking about only a few thousand last-gen GPUs. With current-gen GPUs maybe closer to ~1,000, and with Rubin even fewer I will be damned if I could get my hands on one. That's the part I think people are missing when they extrapolate token demand into enormous infrastructure or AI revenue. Yes usage will explode. But if the cost per unit collapses, the revenue doesn't necessarily go up with it. You can't simultaneously argue that intelligence becomes so cheap that everyone uses enormous amounts of it, while also assuming customers will somehow spend trillions of dollars a year consuming it. There is no obvious $1T customer-facing AI revenue number at the end of this rainbow in the short/medium term. The average person isn't going to spend anything remotely comparable to what they spend on a car every year for an AI service. Even businesses have budgets now, huge demand doesn't matter if the willingness to pay isn't there. The only path I can see to numbers like that is AI consuming existing business domains, even then it's very thin. Say SaaS + legal + consulting + BPO + various other service industries collectively represent something like $10-20T globally. Even if AI eventually replaces an enormous portion of that, it's probably not doing so at the same price. Why would customers switch otherwise? Either the AI product has to be dramatically better, which is difficult for mature workflows, or dramatically cheaper which is much more plausible. If it replaces $10-20T of existing services at roughly 1/10th or 1/100th (more likely) the cost, then you're looking at maybe a ~$1T AI revenue opportunity after replacing an absurdly large fraction of the existing service economy. Who are now unemployed and can't pay for shit. And that's before competition. I think it's crazy to assume AI companies won't compete aggressively on price. As capabilities diffuse, smaller models catch up, inference hits pareto frontier the open-source alternatives have already improved and caught up, margins on routine intelligence should compress "hard" (emphasis on "hard"). We've already seen how difficult adoption can be even when the technology looks impressive on paper. Cheap here means 100x cheaper for 10x more demand that's a net 10x loss before any software or hardware optimizations. So yes, I completely agree that cheap intelligence can bring an enormous amount of new usage. "I just don't think usage means revenue." (you can plaster it on a wall if you want to, "usage doesn't mean revenue", if you want to find that out I have foss software bridge to sell) The PC analogy actually reinforces this if you really think about it. Compute became "vastly more useful" while the cost per unit of compute collapsed. Society captured enormous value, but all computer companies are literal failing giants without the AI hype. Value got caught by people who provided productionization. Now if people expect AI to self productize itself I am happy to tell your try it. We all saw how OpenAI fell behind Anthropic because they thought that would work... Google couldn't productize the search, instead they sold the eye balls and web-real-estate. Maybe that's the AI business model, but that's not $1T worth given you need to unglue people from other stuff. Unless we get something approaching genuine ASI producing so much additional economic value that entirely new trillions, I don't see a path to $1-2T in direct AI revenue from customers. The market simply can't absorb that level of spending. Demand can be effectively infinite at the right price. But I think people are delusional on HN and SF if they think that number is in Trillions like the investments seem to suggest. I am not saying Nvidia will fall tomorrow but someone will have to pull the breaks before this car goes to hell.
- ef33d 2mo ago
  "Yeah, I think there's a tendency to underestimate how much demand is still gated behind cost constraints. The market for this is HUGE." This is just hyperbolic nonsense. There is a desire from a certain group of people of make-believe - doesn't mean the 'demand' is actually real given the economics.
ttoinou 2mo ago
There is no paradox, simply (a/b) increasing tells you nothing about a nor b. Jevons only “destroys” the (wrong) intuition that total b would decrease
- ithkuil 2mo ago
  "paradox" is an overloaded term. Jevons paradox is a veridical paradox, which, as you said, means that it's a true statement that merely looks wrong because it is counterintuitive. I know that some people think that the word "paradox" should be only used to refer to antinomy paradoxes which are often called "true paradoxes" (such as "this sentence is false") which lead to a contradiction without requiring a flaw in reasoning.
  - netcoyote 2mo ago
    Thank you for a great comment; I learned two new words today!
reticulates 2mo ago
I don’t think this line of thinking is particularly robust because it ignores how AI is being used and where the resource usage is coming from. Right now there are a small number of very very resource intensive use cases that are being subsidized by OpenAI and Anthropic. There are people generating millions of lines of code because it’s basically free at the point of use, despite the code producing very little value. Anthropic and OpenAI frequently “reset” customer limits to allow them to use even more resources at no additional cost. The majority of use cases across business are not generating millions of lines of code per employee. The majority of businesses need just a little bit of automation to radically improve the way they operate. A software engineer making endless projects because it’s free to do so might use hundreds of billions of tokens per year, but an entire manufacturing business could be revolutionized with a few million tokens per year. I think 2 things can be true: 1. There is very little penetration of AI across the economy and huge room to grow in the number of businesses deriving economic value from AI 2. The compute usage today is vastly overrepresented by usage outliers who are not paying the cost of their usage and will stop when forced to pay the cost We could see AI usage 10x while seeing compute decrease 10x if the type of usage shifts. Most businesses just need smarter macros.
- zozbot234 2mo ago
  > Anthropic and OpenAI frequently “reset” customer limits to allow them to use even more resources at no additional cost. Surely this applies to fixed-price subscriptions, not per-token spend? Large enterprises (the "very very resource intensive" large-scale users) have to pay per token.
  - reticulates 2mo ago
    A lot of companies avoid paying for usage by encouraging their employees to use individual subscriptions. Outside of the short lived tokenmaxxing fever dream, enterprises are conscious of their usage with companies like Uber and Amazon reigning in their usage massively and companies like Ramp building their own routers for cost minimization. Facebook is reportedly the company that spent $500 million in a single month on tokens. There are individual non-enterprise users rotating multiple subscriptions incurring $10k+ in tokens per subscription. Facebook’s $500 million month… is equivalent to ~10k individual subscriptions which could be as little as a few thousand of the heaviest users. That’s $500 million when billed on usage, or ~$2 million on plans. The reason resets are such a big deal (people have set up websites to track them, tweets announcing them get millions of impressions) is because there are huge numbers of users pushing their plan limits every single day. If there was huge demand from usage-based customers (the large enterprises) that OpenAI and Anthropic couldn’t meet, they wouldn’t be handing out resets like candy. I think a realistic belief is that Anthropic and OpenAI have vastly overstated demand and are using resets as a way to keep usage artificially inflated at a substantial financial cost. I’d guess fixed price plan users make up at least 95% of usage.

kemiller 2mo ago

Yeah, I think there's a tendency to underestimate how much demand is still gated behind cost constraints. The market for this is HUGE. The PC era, call it 1975-2005, was one of the greatest wealth creation events in history, was characterized by the cost of the underlying commodity dropping mercilessly for the whole time. Each time it did, the space of problem you could solve with a PC would increase, to the point that by the end, they were both replacing mainframes and powering users who do nothing but chat and post cat pictures. Could there be a correction in the short run? Quite possibly. I think an underestimated last mile problem is just the massive weight of bureaucracy and human process inertia. But in the long run, cheap, efficient intelligence is a new engineering capability that we've just begun to even explore.

GiorgioG 2mo ago
> The market for this is HUGE. Source(s)?
- formerly_proven 2mo ago
  The backlog of every software team on the planet being anywhere between 1 and 100 years long at human burn rates.
minraws 2mo ago
> how much demand is still gated behind cost constraints. The market for this is HUGE. I think this misses the actual limits here. The problem isn't demand it's, "how much people are willing to spend on it". Cheap AI has to be served on cheap compute, and if inference gets cheap enough to unlock massive usage numbers, by definition it also doesn't require anywhere near as much infrastructure per unit of demand. Take DeepSeek serving ~100T tokens/day, depending on workload and utilization, you're potentially talking about only a few thousand last-gen GPUs. With current-gen GPUs maybe closer to ~1,000, and with Rubin even fewer I will be damned if I could get my hands on one. That's the part I think people are missing when they extrapolate token demand into enormous infrastructure or AI revenue. Yes usage will explode. But if the cost per unit collapses, the revenue doesn't necessarily go up with it. You can't simultaneously argue that intelligence becomes so cheap that everyone uses enormous amounts of it, while also assuming customers will somehow spend trillions of dollars a year consuming it. There is no obvious $1T customer-facing AI revenue number at the end of this rainbow in the short/medium term. The average person isn't going to spend anything remotely comparable to what they spend on a car every year for an AI service. Even businesses have budgets now, huge demand doesn't matter if the willingness to pay isn't there. The only path I can see to numbers like that is AI consuming existing business domains, even then it's very thin. Say SaaS + legal + consulting + BPO + various other service industries collectively represent something like $10-20T globally. Even if AI eventually replaces an enormous portion of that, it's probably not doing so at the same price. Why would customers switch otherwise? Either the AI product has to be dramatically better, which is difficult for mature workflows, or dramatically cheaper which is much more plausible. If it replaces $10-20T of existing services at roughly 1/10th or 1/100th (more likely) the cost, then you're looking at maybe a ~$1T AI revenue opportunity after replacing an absurdly large fraction of the existing service economy. Who are now unemployed and can't pay for shit. And that's before competition. I think it's crazy to assume AI companies won't compete aggressively on price. As capabilities diffuse, smaller models catch up, inference hits pareto frontier the open-source alternatives have already improved and caught up, margins on routine intelligence should compress "hard" (emphasis on "hard"). We've already seen how difficult adoption can be even when the technology looks impressive on paper. Cheap here means 100x cheaper for 10x more demand that's a net 10x loss before any software or hardware optimizations. So yes, I completely agree that cheap intelligence can bring an enormous amount of new usage. "I just don't think usage means revenue." (you can plaster it on a wall if you want to, "usage doesn't mean revenue", if you want to find that out I have foss software bridge to sell) The PC analogy actually reinforces this if you really think about it. Compute became "vastly more useful" while the cost per unit of compute collapsed. Society captured enormous value, but all computer companies are literal failing giants without the AI hype. Value got caught by people who provided productionization. Now if people expect AI to self productize itself I am happy to tell your try it. We all saw how OpenAI fell behind Anthropic because they thought that would work... Google couldn't productize the search, instead they sold the eye balls and web-real-estate. Maybe that's the AI business model, but that's not $1T worth given you need to unglue people from other stuff. Unless we get something approaching genuine ASI producing so much additional economic value that entirely new trillions, I don't see a path to $1-2T in direct AI revenue from customers. The market simply can't absorb that level of spending. Demand can be effectively infinite at the right price. But I think people are delusional on HN and SF if they think that number is in Trillions like the investments seem to suggest. I am not saying Nvidia will fall tomorrow but someone will have to pull the breaks before this car goes to hell.
ef33d 2mo ago
"Yeah, I think there's a tendency to underestimate how much demand is still gated behind cost constraints. The market for this is HUGE." This is just hyperbolic nonsense. There is a desire from a certain group of people of make-believe - doesn't mean the 'demand' is actually real given the economics.

ttoinou 2mo ago

There is no paradox, simply (a/b) increasing tells you nothing about a nor b. Jevons only “destroys” the (wrong) intuition that total b would decrease

ithkuil 2mo ago
"paradox" is an overloaded term. Jevons paradox is a veridical paradox, which, as you said, means that it's a true statement that merely looks wrong because it is counterintuitive. I know that some people think that the word "paradox" should be only used to refer to antinomy paradoxes which are often called "true paradoxes" (such as "this sentence is false") which lead to a contradiction without requiring a flaw in reasoning.
- netcoyote 2mo ago
  Thank you for a great comment; I learned two new words today!

reticulates 2mo ago

I don’t think this line of thinking is particularly robust because it ignores how AI is being used and where the resource usage is coming from. Right now there are a small number of very very resource intensive use cases that are being subsidized by OpenAI and Anthropic. There are people generating millions of lines of code because it’s basically free at the point of use, despite the code producing very little value. Anthropic and OpenAI frequently “reset” customer limits to allow them to use even more resources at no additional cost. The majority of use cases across business are not generating millions of lines of code per employee. The majority of businesses need just a little bit of automation to radically improve the way they operate. A software engineer making endless projects because it’s free to do so might use hundreds of billions of tokens per year, but an entire manufacturing business could be revolutionized with a few million tokens per year. I think 2 things can be true: 1. There is very little penetration of AI across the economy and huge room to grow in the number of businesses deriving economic value from AI 2. The compute usage today is vastly overrepresented by usage outliers who are not paying the cost of their usage and will stop when forced to pay the cost We could see AI usage 10x while seeing compute decrease 10x if the type of usage shifts. Most businesses just need smarter macros.

zozbot234 2mo ago
> Anthropic and OpenAI frequently “reset” customer limits to allow them to use even more resources at no additional cost. Surely this applies to fixed-price subscriptions, not per-token spend? Large enterprises (the "very very resource intensive" large-scale users) have to pay per token.
- reticulates 2mo ago
  A lot of companies avoid paying for usage by encouraging their employees to use individual subscriptions. Outside of the short lived tokenmaxxing fever dream, enterprises are conscious of their usage with companies like Uber and Amazon reigning in their usage massively and companies like Ramp building their own routers for cost minimization. Facebook is reportedly the company that spent $500 million in a single month on tokens. There are individual non-enterprise users rotating multiple subscriptions incurring $10k+ in tokens per subscription. Facebook’s $500 million month… is equivalent to ~10k individual subscriptions which could be as little as a few thousand of the heaviest users. That’s $500 million when billed on usage, or ~$2 million on plans. The reason resets are such a big deal (people have set up websites to track them, tweets announcing them get millions of impressions) is because there are huge numbers of users pushing their plan limits every single day. If there was huge demand from usage-based customers (the large enterprises) that OpenAI and Anthropic couldn’t meet, they wouldn’t be handing out resets like candy. I think a realistic belief is that Anthropic and OpenAI have vastly overstated demand and are using resets as a way to keep usage artificially inflated at a substantial financial cost. I’d guess fixed price plan users make up at least 95% of usage.