4 ms·

> I doubt we will see unused GPU capacity I'd argue we very certainly will. Companies are gobbling up GPUs like there's no tomorrow, assuming demand will remai

by Mo3 1y ago

> I doubt we will see unused GPU capacity I'd argue we very certainly will. Companies are gobbling up GPUs like there's no tomorrow, assuming demand will remain stable and continue growing indefinitely. Meanwhile LLM fatigue has started to set in, models are getting smaller and smaller and consumer hardware is getting better and better. There's no way this won't end up with a lot of idle GPUs.

xadhominemx 1y ago
Test time compute has made consumption highly elastic. More compute = better results. Marginal cost of running these GPUs when they would otherwise be idle is relatively very low. It will be utilized.
delusional 1y ago
> There's no way this won't end up with a lot of idle GPUs. Nvidia is betting the farm on reinventing GPU compute every 2 years. The GPUs wont end up idle, because they will end up in landfills. Do I believe that's likely, no, but it is what I believe Nvidia is aiming for.
Workaccount2 1y ago
>Meanwhile LLM fatigue has started to set in Has it? I think there is this compulsion to think that LLMs are made for senior devs, and if devs are getting wary of LLMs, the experiment is over. I'm not a programmer, my day job isn't tech, and the only people I know who express discontent with LLMs are a few of programmer friends I have. Which I get, but everyone else is using them gleefully for all manner of stuff. And now I am seeing the very first inklings of completely non-technical people making bespoke applets for themselves. From OpenAI, programming is ~4% of chatGPTs usage. That's 96% being used for other stuff. I don't see any realistic or grounded forecast that includes a diminishing demand for compute. We're still at the tip of adoption...
- Mistletoe 1y ago
  You should get on Reddit, people hate AI with a passion there. People I meet in real life hate it also. I think the public actually hates AI more than it should now.
  - chii 1y ago
    It's a pretty biased sample. Not to mention that people who are neutral and is just using AI won't be bothered to comment. So you only ever see one extreme or another.
  - arkmm 1y ago
    The irony of this is so much of Reddit comments these days are AI generated.
  - Workaccount2 1y ago
    I spent 13 years chronically on reddit before stumbling into a exit hatch of the bubble chamber. Those people (well really it's teens and college kids) live on reddit, they are so far from an accurate representation of reality its insane.
    - transcriptase 1y ago
      Worse when you find out there’s a couple dozen of the same moderators running nearly all the top 500 subreddits.
      - hnhg 1y ago
        That makes some sense. Are they paid for this?
        GauntletWizard 1y ago
        By reddit? No. By the users? No. But they are paid.
        1y ago
- pessimizer 1y ago
  > From OpenAI, programming is ~4% of chatGPTs usage. That's 96% being used for other stuff. I think it's important to remember that a good bunch of this is going to be people using it as an artificial friend, which is not really productive. Really that's destructive, because in that time you could be creating a relationship with an actual person instead of a context soon to be deleted. But on the other hand, some people are using it as an artificial smart friend, asking it questions that they would be embarrassed to ask to other people, and learning. That could be a very good thing, but it's only as good as the people who own and tune the LLMs are. Sadly, they seem to be a bunch of oligarchs who are half sociopaths and half holy warriors. As for compute, people using it as an artificial friend are either going to have a low price ceiling, or in an even worse case scenario they are not and it's going to be like gambling addiction.
  - baq 1y ago
    Productive or destructive, demand is there, so it isn’t late bubble. It’s still early. (Which is scary, I’ll readily admit.)
    - pessimizer 1y ago
      But demand isn't there (or rather, proven to be there.) Demand is measured in dollars, and right now VC is paying. This is peak bubble - farthest distance from valuation and income.
      - yeasku 1y ago
        Even if its there, will it be in 10 years?
      - baq 1y ago
        Microsoft and meta are not VCs and they’re spending money on data centers like there’s no tomorrow, doesn’t seem very low demand.
brazukadev 1y ago
This. I just found out that for my MCP needs, Qwen3 4B running local is good enough! So I just stopped using Gemini API.
idiotsecant 1y ago
Your bet is that people will simply use less compute, for the first time in the history of the human race?
- Mo3 1y ago
  No, mostly less external compute
mg 1y ago
Look at the human body. 2% of it is dedicated to thinking. My guess is that as a species, we will turn a similar percentage of our environment into thinking matter. If there are a billion houses on planet earth, 2% of it are 20 million datacenters we still have to build.
- wussboy 1y ago
  An analogy is not proof. It is not even evidence.
goalieca 1y ago
What’s the lifetime of these things once they’ve been running hot for 2-3 years

by Mo3 1y ago

xadhominemx 1y ago

Test time compute has made consumption highly elastic. More compute = better results. Marginal cost of running these GPUs when they would otherwise be idle is relatively very low. It will be utilized.

delusional 1y ago

> There's no way this won't end up with a lot of idle GPUs. Nvidia is betting the farm on reinventing GPU compute every 2 years. The GPUs wont end up idle, because they will end up in landfills. Do I believe that's likely, no, but it is what I believe Nvidia is aiming for.

Workaccount2 1y ago

>Meanwhile LLM fatigue has started to set in Has it? I think there is this compulsion to think that LLMs are made for senior devs, and if devs are getting wary of LLMs, the experiment is over. I'm not a programmer, my day job isn't tech, and the only people I know who express discontent with LLMs are a few of programmer friends I have. Which I get, but everyone else is using them gleefully for all manner of stuff. And now I am seeing the very first inklings of completely non-technical people making bespoke applets for themselves. From OpenAI, programming is ~4% of chatGPTs usage. That's 96% being used for other stuff. I don't see any realistic or grounded forecast that includes a diminishing demand for compute. We're still at the tip of adoption...

Mistletoe 1y ago
You should get on Reddit, people hate AI with a passion there. People I meet in real life hate it also. I think the public actually hates AI more than it should now.
- chii 1y ago
  It's a pretty biased sample. Not to mention that people who are neutral and is just using AI won't be bothered to comment. So you only ever see one extreme or another.
- arkmm 1y ago
  The irony of this is so much of Reddit comments these days are AI generated.
- Workaccount2 1y ago
  I spent 13 years chronically on reddit before stumbling into a exit hatch of the bubble chamber. Those people (well really it's teens and college kids) live on reddit, they are so far from an accurate representation of reality its insane.
  - transcriptase 1y ago
    Worse when you find out there’s a couple dozen of the same moderators running nearly all the top 500 subreddits.
    - hnhg 1y ago
      That makes some sense. Are they paid for this?
      - GauntletWizard 1y ago
        By reddit? No. By the users? No. But they are paid.
        1y ago
pessimizer 1y ago
> From OpenAI, programming is ~4% of chatGPTs usage. That's 96% being used for other stuff. I think it's important to remember that a good bunch of this is going to be people using it as an artificial friend, which is not really productive. Really that's destructive, because in that time you could be creating a relationship with an actual person instead of a context soon to be deleted. But on the other hand, some people are using it as an artificial smart friend, asking it questions that they would be embarrassed to ask to other people, and learning. That could be a very good thing, but it's only as good as the people who own and tune the LLMs are. Sadly, they seem to be a bunch of oligarchs who are half sociopaths and half holy warriors. As for compute, people using it as an artificial friend are either going to have a low price ceiling, or in an even worse case scenario they are not and it's going to be like gambling addiction.
- baq 1y ago
  Productive or destructive, demand is there, so it isn’t late bubble. It’s still early. (Which is scary, I’ll readily admit.)
  - pessimizer 1y ago
    But demand isn't there (or rather, proven to be there.) Demand is measured in dollars, and right now VC is paying. This is peak bubble - farthest distance from valuation and income.
    - yeasku 1y ago
      Even if its there, will it be in 10 years?
    - baq 1y ago
      Microsoft and meta are not VCs and they’re spending money on data centers like there’s no tomorrow, doesn’t seem very low demand.

brazukadev 1y ago

This. I just found out that for my MCP needs, Qwen3 4B running local is good enough! So I just stopped using Gemini API.

idiotsecant 1y ago

Your bet is that people will simply use less compute, for the first time in the history of the human race?

Mo3 1y ago
No, mostly less external compute

mg 1y ago

Look at the human body. 2% of it is dedicated to thinking. My guess is that as a species, we will turn a similar percentage of our environment into thinking matter. If there are a billion houses on planet earth, 2% of it are 20 million datacenters we still have to build.

wussboy 1y ago
An analogy is not proof. It is not even evidence.

goalieca 1y ago

What’s the lifetime of these things once they’ve been running hot for 2-3 years