5 ms·

Compute has already lost value for me. Six months ago I thought you needed a 1T+ model to be useful coding. Now I am able to get by just fine with a 27b model.

Compute has already lost value for me. Six months ago I thought you needed a 1T+ model to be useful coding. Now I am able to get by just fine with a 27b model. I see two factors converging to cause a collapse of this house of cards: 1. People are realizing that what they need isn't more general intelligence, it's more specialization. A small but well tuned coding model, a small but well tuned customer service model, a small but well tuned document explorer. 2. Specialized hardware - TPUs and NPUs - especially coming out of china. The latest GLM model was trained and runs on Huawei hardware. Nvidia is only worth so much because they are the biggest and best provider of the kind of compute needed to run llms, but the export bans mean china has a lot of incentive to topple that monopoly. The amount of compute we need to do the things llms do is falling rapidly, the number of people who can provide that compute is rising.

SleightOfHand 1mo ago
[dead]
- kennywinker 1mo ago
  OpenAI dropped sora because it was costing them ridiculous amounts of money and earning them very little. They determined that the market can't support the cost of generating video. Without a material change in the market (more buyers, vastly cheaper generation), it's unlikely a different company could make that work. More buyers isn't likely to happen, so that leaves vastly cheaper generation - something that would cause nvidia's value to collapse if it happened.
  - fc417fc802 1mo ago
    > the market can't support the cost of generating video. I'd suggest that's only the case given the current quality of output. Media is incredibly expensive to produce. A model capable of sufficiently high quality could charge prices that are absurd by today's standards.
    - gbear605 1mo ago
      It’s a very small set of buyers that are in that price range. Total annual domestic box office revenue is like $10 billion, maybe $50 billion for global TV and film. And that’s revenue, not profit, and a lot of costs are going to marketing, not to filming and casting. That’s a lot of money, but it’s not the scale that OpenAI and Anthropic are at. Video generation would only make sense at that scale if it was targeting individual consumers, but then it’d need to cost something that consumers are willing to pay - which practically is probably a few hundred per year at most among US consumers, and much less globally, so again it doesn’t solve for the size of the AI companies. I don’t see a way that video generation becomes a big industry without making generation much much cheaper.
      - fc417fc802 1mo ago
        Aren't these two largely separate questions? Viability versus if a given incumbent has interest in a market of a given size. With the combination of (at minimum) streaming platforms, the box office, and advertisements video and audio generation would be viable at a remarkably high price point (as compared to the current token prices for other sorts of things). And as the price comes down presumably the market would grow larger - by how much I have no idea but there are certainly a great deal of currently underserved niche markets.
        ethbr1 1mo ago
        I think the idea that people are willing to consume an even more slopful Netflix is dubious. There's has already been a decade of complaints about trash content in excess of available viewing hours. The thing with media consumption is that there exist a finite number of eyeball-hours available, and it's a zero sum game against other non-media consumers.
- indigodaddy 1mo ago
  oAI isn't anywhere near close to fucked as long as their models are head and shoulders above even the very best open models in terms of tool calling and rock solid stability/reliability for agents/coding harnesses. Which, they are right now and we'll see if open models actually catch up in that regard. Even the "best" open models pale in comparison with tool calling and general "prompt and go do something else for an hour" reliability that we have with GPT models. With GPT models, streaming rarely stops unexpectedly. You almost never have to constantly nudge them along, etc. Granted with open models all of this can vary depending on the provider, and perhaps open models/protocols/APIs/harnesses aren't well enough aligned, but OpenAI models just seem to work without constant (or hardly any) wrinkles and with almost any harness/agent.
  - InsideOutSanta 1mo ago
    >oAI isn't anywhere near close to fucked as long as their models are head and shoulders above even the very best open models in terms of tool calling and rock solid stability/reliability for agents/coding harnesses That's already not the case today. If you sat me in front of an LLM and told me to figure out if I'm working with K3 or Astra, I could probably do it, but it would take some work to be certain.
    - phoghed 1mo ago
      All we do with these things is some work though
    - indigodaddy 1mo ago
      Well I could for sure. I guess a lot of this is indeed very anecdotal.
  - williamse 1mo ago
    [flagged]
- usefulcat 1mo ago
  There would also need to exist sufficient demand for video, which hasn’t happened yet.
  - tiznow 1mo ago
    I would also add: at profit-taking pricing
- chpatrick 1mo ago
  Minimax H3 works pretty great and you can run it on a 3090.
zer00eyz 1mo ago
If you reshuffle your argument, and apply the same facts you get to a similar conclusion but with a drastically different spin. > it's more specialization China, constrained by hardware, and talent (not to slight the Chinese, but they are limited to domestic resources - and much of the US effort is very international). They did, what the Chinese do, and optimized the process of production, and drastically lowered the cost of development of their models. Cheeper to build, cheaper to run is just good economics. Meanwhile in the us, we have open AI doing "experiments" - it looks like the costs around the hugging face hack are going to be about the same as China would spend on building out one of their smaller efforts (several million dollars). (Depending on whos numbers you trust, the fact that I can even make this claim should make you raise an eyebrow). Go back to the 80s' and "expert systems" - most people will tell you that for their time, they were amazing, and useful. People would have loved to have more of them but they were so cost prohibitive that we all but abandoned them for serious use. The US frontier labs seem to have forgotten this lesson and their calls to "slow down" look like an excuse to "cut the waste so we can move to making money".
sellmesoap 1mo ago
I think what will keep the industry afloat, all else failing, is the surveillance industry! Nothing like a fat reoccurring cheque from the government to check if little Jimmy is committing thought crime!
pilooch 1mo ago
That's unless the code produced in the future is much more complex than today's.
- zahlman 1mo ago
  Sure, but it would be actively bad to make the code more complex simply because we have machinery that helps us deal with the complexity. A big part of how people assess the models' coding capability is whether they create needless, incidental complexity.
  - vasco 1mo ago
    That's like saying it'd be actively bad to make the code more resource intensive simply because we have machinery that helps us deal with the extra requirements. And as we know as computers got more powerful code didn't get lighter. If it can, it will.
    - detourdog 1mo ago
      Using more resources than necessary is actually waste.
      - vasco 1mo ago
        That's my point
        zahlman 1mo ago
        I'm confused. You started out with "That's like saying", but you appear to agree with me completely.
        vasco 1mo ago
        What I mean is we have evidence that if it's easy and possible people will do it. There's no need to predict anything. That it is bad is besides the fact, although obviously less performant software is worse, its easier to create.
larodi 1mo ago
They need the right harness and either your help it auto produces in time enough content to further improve.
- kennywinker 1mo ago
  A classic big ai talking point. Color me skeptical.
vunderba 1mo ago
> People are realizing that what they need isn't more general intelligence, it's more specialization. A small but well tuned coding model... It’s not quite as simple as that. Several studies have shown the opposite: models trained on more diverse knowledge tend to cross-pollinate across domains. So a more generalized model can actually perform better than a specialized one. That’s why you’re not seeing tons of tiny models (one for Python, one for Pascal, one for Rust, etc).
- kennywinker 1mo ago
  This is definitely the position of the big ai companies. But it doesn't match my experience. Qwen3.8 27b is clearly smarter at coding than MANY bigger models. gpt-oss-120b for example, is almost 4x the size, and performs way worse at coding tasks. It's clear to me that you can build small models that work well at specific tasks. Python vs Rust is probably too fine grained a way to build a model. Coding in general seems like a better target. There will always be a place for large generalist models, no doubt. But I think that place is much smaller than the big ai companies are counting on.
  - vunderba 1mo ago
    I think we’re in agreement. I make heavy use of smaller local models on a daily basis (Qwen3-VL for auto-captioning images, Gemma3:27b for some translation work, etc.). Gemma3:27b is a good example of a very capable general purpose multimodal model and has handled almost everything I've thrown at it from sentiment analysis to documentation writing. I suppose I was drawing a distinction between specialized and general intelligence versus small and large. I don’t think those are necessarily mutually exclusive.
  - ac29 1mo ago
    gpt-oss-120b only has 5B active parameters, so its not surprising Qwen3.8 27B outperforms it (Qwen3.8 is also ~13 months newer, which is forever in LLMs)
    - kennywinker 1mo ago
pvab3 1mo ago
I've been thinking about that and that's why Nvidia's prices are surprising to me. Investors should know that better than me so there must be something I don't know
- detourdog 1mo ago
  It’s really hard to know when the large tech companies have so many shares owned by a single figure. They can use margin loans and options to create the appearance of demand.
- torginus 1mo ago
  Money is not real. Just like how I could borrow $100 from you, and you could borrow $100 from me, and we'd be $200 in debt in total, I could buy 10% of shares in your company for $1m and you could buy 10% shares in mine, we'd have 2 companies worth $20m together in total.
e_y_ 1mo ago
LLMs needing less compute would actually be a good thing for Nvidia due to Jevons paradox. Right now token costs are an impediment to using AI more broadly, and more efficient models would help adoption in cases where AI has proven to be useful, like coding. https://en.wikipedia.org/wiki/Jevons_paradox https://en.wikipedia.org/wiki/Jevons_paradox
- janalsncm 1mo ago
  Jevon’s paradox is a common talking point but it is not a law of nature. LED lightbulbs use 80% less energy than incandescent but you don’t see people using 5x more lights on their homes. The overall energy used to light homes has decreased. And even if compute demand were perfectly elastic it’s only a good thing insofar as it drives demand for new Nvidia hardware. If tokens can be served from Apple hardware or Google hardware or Huawei hardware that doesn’t help Nvidia.
  - kennywinker 1mo ago
    > LED lightbulbs use 80% less energy than incandescent but you don’t see people using 5x more lights on their homes. I mean… some homes definitely do. You must have seen those houses that are all lit up front the outside by lawn mounted spotlights.
  - tonyarkles 1mo ago
    As I look up and see three lightbulbs side-by-side over my desk and another one in the lamp on the desk… which is dramatically more light than the old 100W bulb used but also lower energy consumption.
    - omgwtfbyobbq 1mo ago
      That's the rebound effect, and is fairly common. Jevons is a relatively uncommon edge case.
  - ethbr1 1mo ago
    The more apt analogy is between pre-electric and post-electric lighting (~1850 to 1920), when efficiency of light generation rapidly increased, restabilizing at ~222x as much light for the same unit of human labor. [0] Across that period, first world countries massively increased their demand for light. Further efficiency increases only matter to individual choices if they take a use case from {economically impossible} to {economically possible}. I'd offer that by the 1920s, most goings-on in first world urban environments were no longer price constrained in terms of their light usage. [0] See table 1.4, p21 https://www.nber.org/system/files/chapters/c6064/c6064.pdf https://www.nber.org/system/files/chapters/c6064/c6064.pdf
keeda 1mo ago
This is the right kind of analysis, but we can look broader. Both the demand and supply situations are a lot more extreme and dynamic than appears at first glance. E.g. to your points: 1. Yes, smaller models will become more popular, especially as the tokenmaxxing trend dies down and people start stretching their budgets farther. That is a downward pressure on demand. But along the same dimension, consider that currently only about 40 - 60% of the world uses AI for only about 5 - 15% of their work hours. That means there is still 2x growth from users and 7x - 20x growth from the rest of the work hours left to capture! That is 14x - 40x more demand. Then consider that agentic tasks require multiples more tokens, and that is the kind of usage that is most likely to be deployed, and also the kind of usage that is the least used right now. That's another huge multiple to be tacked on. And the entire AI industry has been lamenting the extreme compute crunch they're facing (and also why Claude has 9's comparable to GitHub; whereas OpenAI has been chugging along because Altman was OK being called a "podcasting bro" while desperately scrounging for compute years in advance.) Nvidia's meteoric rise is entirely due to this kind of exploding demand with extremely limited supply. 2. Competing hardware is definitely a threat, but it has its own hurdles. Because the real bottleneck is not Nvidia, it's TSMC. Pretty much all demand for all chips in all devices in all the world flow to, like, 3 companies in the world that actually fabricate them, and TSMC is the biggest. And the supply is extremely tight, as the exploding costs of electronics clearly shows. So now TSMC will of course try to keep all its customers happy, but it will inevitably be forced to choose which ones it will keep happiest. And those will be the customers who can pay it the most. And that would be the one with all the money from its de facto status as a monopoly (and possibly even a monopsony)... Which would be Nvidia ;-) So yes, compute per task is falling rapidly... but it's barely a dent in the humongous total addressable demand, and the amount of hardware to support that compute is still very constrained, and most of that supply will likely flow through Nvidia.
- kennywinker 1mo ago
  > But along the same dimension, consider that currently only about 40 - 60% of the world uses AI for only about 5 - 15% of their work hours. Ah yes, i am constantly lamenting that my barista isn’t using ai enough ;) Hopefully you’ve adjusted your ceiling numbers to account for the large amount of people who can’t afford to pay for llms, and will never be able to pay, and aren’t worth it to advertise to since they can afford very little

SleightOfHand 1mo ago

[dead]

kennywinker 1mo ago
OpenAI dropped sora because it was costing them ridiculous amounts of money and earning them very little. They determined that the market can't support the cost of generating video. Without a material change in the market (more buyers, vastly cheaper generation), it's unlikely a different company could make that work. More buyers isn't likely to happen, so that leaves vastly cheaper generation - something that would cause nvidia's value to collapse if it happened.
- fc417fc802 1mo ago
  > the market can't support the cost of generating video. I'd suggest that's only the case given the current quality of output. Media is incredibly expensive to produce. A model capable of sufficiently high quality could charge prices that are absurd by today's standards.
  - gbear605 1mo ago
    It’s a very small set of buyers that are in that price range. Total annual domestic box office revenue is like $10 billion, maybe $50 billion for global TV and film. And that’s revenue, not profit, and a lot of costs are going to marketing, not to filming and casting. That’s a lot of money, but it’s not the scale that OpenAI and Anthropic are at. Video generation would only make sense at that scale if it was targeting individual consumers, but then it’d need to cost something that consumers are willing to pay - which practically is probably a few hundred per year at most among US consumers, and much less globally, so again it doesn’t solve for the size of the AI companies. I don’t see a way that video generation becomes a big industry without making generation much much cheaper.
    - fc417fc802 1mo ago
      Aren't these two largely separate questions? Viability versus if a given incumbent has interest in a market of a given size. With the combination of (at minimum) streaming platforms, the box office, and advertisements video and audio generation would be viable at a remarkably high price point (as compared to the current token prices for other sorts of things). And as the price comes down presumably the market would grow larger - by how much I have no idea but there are certainly a great deal of currently underserved niche markets.
      - ethbr1 1mo ago
        I think the idea that people are willing to consume an even more slopful Netflix is dubious. There's has already been a decade of complaints about trash content in excess of available viewing hours. The thing with media consumption is that there exist a finite number of eyeball-hours available, and it's a zero sum game against other non-media consumers.
indigodaddy 1mo ago
oAI isn't anywhere near close to fucked as long as their models are head and shoulders above even the very best open models in terms of tool calling and rock solid stability/reliability for agents/coding harnesses. Which, they are right now and we'll see if open models actually catch up in that regard. Even the "best" open models pale in comparison with tool calling and general "prompt and go do something else for an hour" reliability that we have with GPT models. With GPT models, streaming rarely stops unexpectedly. You almost never have to constantly nudge them along, etc. Granted with open models all of this can vary depending on the provider, and perhaps open models/protocols/APIs/harnesses aren't well enough aligned, but OpenAI models just seem to work without constant (or hardly any) wrinkles and with almost any harness/agent.
- InsideOutSanta 1mo ago
  >oAI isn't anywhere near close to fucked as long as their models are head and shoulders above even the very best open models in terms of tool calling and rock solid stability/reliability for agents/coding harnesses That's already not the case today. If you sat me in front of an LLM and told me to figure out if I'm working with K3 or Astra, I could probably do it, but it would take some work to be certain.
  - phoghed 1mo ago
    All we do with these things is some work though
  - indigodaddy 1mo ago
    Well I could for sure. I guess a lot of this is indeed very anecdotal.
- williamse 1mo ago
  [flagged]
usefulcat 1mo ago
There would also need to exist sufficient demand for video, which hasn’t happened yet.
- tiznow 1mo ago
  I would also add: at profit-taking pricing
chpatrick 1mo ago
Minimax H3 works pretty great and you can run it on a 3090.

zer00eyz 1mo ago

If you reshuffle your argument, and apply the same facts you get to a similar conclusion but with a drastically different spin. > it's more specialization China, constrained by hardware, and talent (not to slight the Chinese, but they are limited to domestic resources - and much of the US effort is very international). They did, what the Chinese do, and optimized the process of production, and drastically lowered the cost of development of their models. Cheeper to build, cheaper to run is just good economics. Meanwhile in the us, we have open AI doing "experiments" - it looks like the costs around the hugging face hack are going to be about the same as China would spend on building out one of their smaller efforts (several million dollars). (Depending on whos numbers you trust, the fact that I can even make this claim should make you raise an eyebrow). Go back to the 80s' and "expert systems" - most people will tell you that for their time, they were amazing, and useful. People would have loved to have more of them but they were so cost prohibitive that we all but abandoned them for serious use. The US frontier labs seem to have forgotten this lesson and their calls to "slow down" look like an excuse to "cut the waste so we can move to making money".

sellmesoap 1mo ago

I think what will keep the industry afloat, all else failing, is the surveillance industry! Nothing like a fat reoccurring cheque from the government to check if little Jimmy is committing thought crime!

pilooch 1mo ago

That's unless the code produced in the future is much more complex than today's.

zahlman 1mo ago
Sure, but it would be actively bad to make the code more complex simply because we have machinery that helps us deal with the complexity. A big part of how people assess the models' coding capability is whether they create needless, incidental complexity.
- vasco 1mo ago
  That's like saying it'd be actively bad to make the code more resource intensive simply because we have machinery that helps us deal with the extra requirements. And as we know as computers got more powerful code didn't get lighter. If it can, it will.
  - detourdog 1mo ago
    Using more resources than necessary is actually waste.
    - vasco 1mo ago
      That's my point
      - zahlman 1mo ago
        I'm confused. You started out with "That's like saying", but you appear to agree with me completely.
        vasco 1mo ago
        What I mean is we have evidence that if it's easy and possible people will do it. There's no need to predict anything. That it is bad is besides the fact, although obviously less performant software is worse, its easier to create.

larodi 1mo ago

They need the right harness and either your help it auto produces in time enough content to further improve.

kennywinker 1mo ago
A classic big ai talking point. Color me skeptical.

vunderba 1mo ago

> People are realizing that what they need isn't more general intelligence, it's more specialization. A small but well tuned coding model... It’s not quite as simple as that. Several studies have shown the opposite: models trained on more diverse knowledge tend to cross-pollinate across domains. So a more generalized model can actually perform better than a specialized one. That’s why you’re not seeing tons of tiny models (one for Python, one for Pascal, one for Rust, etc).

kennywinker 1mo ago
This is definitely the position of the big ai companies. But it doesn't match my experience. Qwen3.8 27b is clearly smarter at coding than MANY bigger models. gpt-oss-120b for example, is almost 4x the size, and performs way worse at coding tasks. It's clear to me that you can build small models that work well at specific tasks. Python vs Rust is probably too fine grained a way to build a model. Coding in general seems like a better target. There will always be a place for large generalist models, no doubt. But I think that place is much smaller than the big ai companies are counting on.
- vunderba 1mo ago
  I think we’re in agreement. I make heavy use of smaller local models on a daily basis (Qwen3-VL for auto-captioning images, Gemma3:27b for some translation work, etc.). Gemma3:27b is a good example of a very capable general purpose multimodal model and has handled almost everything I've thrown at it from sentiment analysis to documentation writing. I suppose I was drawing a distinction between specialized and general intelligence versus small and large. I don’t think those are necessarily mutually exclusive.
- ac29 1mo ago
  gpt-oss-120b only has 5B active parameters, so its not surprising Qwen3.8 27B outperforms it (Qwen3.8 is also ~13 months newer, which is forever in LLMs)
  - kennywinker 1mo ago

pvab3 1mo ago

I've been thinking about that and that's why Nvidia's prices are surprising to me. Investors should know that better than me so there must be something I don't know

detourdog 1mo ago
It’s really hard to know when the large tech companies have so many shares owned by a single figure. They can use margin loans and options to create the appearance of demand.
torginus 1mo ago
Money is not real. Just like how I could borrow $100 from you, and you could borrow $100 from me, and we'd be $200 in debt in total, I could buy 10% of shares in your company for $1m and you could buy 10% shares in mine, we'd have 2 companies worth $20m together in total.

e_y_ 1mo ago

LLMs needing less compute would actually be a good thing for Nvidia due to Jevons paradox. Right now token costs are an impediment to using AI more broadly, and more efficient models would help adoption in cases where AI has proven to be useful, like coding. https://en.wikipedia.org/wiki/Jevons_paradox https://en.wikipedia.org/wiki/Jevons_paradox

janalsncm 1mo ago
Jevon’s paradox is a common talking point but it is not a law of nature. LED lightbulbs use 80% less energy than incandescent but you don’t see people using 5x more lights on their homes. The overall energy used to light homes has decreased. And even if compute demand were perfectly elastic it’s only a good thing insofar as it drives demand for new Nvidia hardware. If tokens can be served from Apple hardware or Google hardware or Huawei hardware that doesn’t help Nvidia.
- kennywinker 1mo ago
  > LED lightbulbs use 80% less energy than incandescent but you don’t see people using 5x more lights on their homes. I mean… some homes definitely do. You must have seen those houses that are all lit up front the outside by lawn mounted spotlights.
- tonyarkles 1mo ago
  As I look up and see three lightbulbs side-by-side over my desk and another one in the lamp on the desk… which is dramatically more light than the old 100W bulb used but also lower energy consumption.
  - omgwtfbyobbq 1mo ago
    That's the rebound effect, and is fairly common. Jevons is a relatively uncommon edge case.
- ethbr1 1mo ago
  The more apt analogy is between pre-electric and post-electric lighting (~1850 to 1920), when efficiency of light generation rapidly increased, restabilizing at ~222x as much light for the same unit of human labor. [0] Across that period, first world countries massively increased their demand for light. Further efficiency increases only matter to individual choices if they take a use case from {economically impossible} to {economically possible}. I'd offer that by the 1920s, most goings-on in first world urban environments were no longer price constrained in terms of their light usage. [0] See table 1.4, p21 https://www.nber.org/system/files/chapters/c6064/c6064.pdf https://www.nber.org/system/files/chapters/c6064/c6064.pdf

keeda 1mo ago

This is the right kind of analysis, but we can look broader. Both the demand and supply situations are a lot more extreme and dynamic than appears at first glance. E.g. to your points: 1. Yes, smaller models will become more popular, especially as the tokenmaxxing trend dies down and people start stretching their budgets farther. That is a downward pressure on demand. But along the same dimension, consider that currently only about 40 - 60% of the world uses AI for only about 5 - 15% of their work hours. That means there is still 2x growth from users and 7x - 20x growth from the rest of the work hours left to capture! That is 14x - 40x more demand. Then consider that agentic tasks require multiples more tokens, and that is the kind of usage that is most likely to be deployed, and also the kind of usage that is the least used right now. That's another huge multiple to be tacked on. And the entire AI industry has been lamenting the extreme compute crunch they're facing (and also why Claude has 9's comparable to GitHub; whereas OpenAI has been chugging along because Altman was OK being called a "podcasting bro" while desperately scrounging for compute years in advance.) Nvidia's meteoric rise is entirely due to this kind of exploding demand with extremely limited supply. 2. Competing hardware is definitely a threat, but it has its own hurdles. Because the real bottleneck is not Nvidia, it's TSMC. Pretty much all demand for all chips in all devices in all the world flow to, like, 3 companies in the world that actually fabricate them, and TSMC is the biggest. And the supply is extremely tight, as the exploding costs of electronics clearly shows. So now TSMC will of course try to keep all its customers happy, but it will inevitably be forced to choose which ones it will keep happiest. And those will be the customers who can pay it the most. And that would be the one with all the money from its de facto status as a monopoly (and possibly even a monopsony)... Which would be Nvidia ;-) So yes, compute per task is falling rapidly... but it's barely a dent in the humongous total addressable demand, and the amount of hardware to support that compute is still very constrained, and most of that supply will likely flow through Nvidia.

kennywinker 1mo ago
> But along the same dimension, consider that currently only about 40 - 60% of the world uses AI for only about 5 - 15% of their work hours. Ah yes, i am constantly lamenting that my barista isn’t using ai enough ;) Hopefully you’ve adjusted your ceiling numbers to account for the large amount of people who can’t afford to pay for llms, and will never be able to pay, and aren’t worth it to advertise to since they can afford very little