3 ms·

I was just thinking a few weeks ago that there is a risk LLMs will bring on a wave of big computer builds similar to crypto mining. For storing all the data, an

I was just thinking a few weeks ago that there is a risk LLMs will bring on a wave of big computer builds similar to crypto mining. For storing all the data, and for calculating all the queries from their users. I'm not familiar with what is required but I do believe that the larger the dataset the better it is, and that specialized GPUs have already been released.

Gormo 3y ago
It's odd to characterize that as a "risk". It seems to follow that increases in demand for computing use cases would in turn generate more demand for the hardware required to implement them.
- INTPenis 3y ago
  The article was about electricity consumption, so in the context of environmental impact, or consumer market impact, it is a risk.
  - Gormo 3y ago
    Consumption of electricity has negligible environmental impact. The relevant discussion here would pertain to considerations around generating power, regardless of what downstream use cases the power is being applied to. Pretty much all activity in modern society is going to consume electricity, and overall demand is not going to be decreasing in the first place, so it seems a bit silly to look at this from the demand side: we're always going to need more and more power, and the focus is properly on how to generate power in a clean and scalable way regardless of what it's being used for. Consumer market impact is an interesting topic, though, if there are massive spikes in demand that could drive prices up for other users of electricity. It will be important to ensure to minimize artificial impediments to the expansion of the supply side to mitigate that risk.
verdverm 3y ago
More data & compute still produces noticeably better results, and probably always will (?) The difference is that the AI field is - rapidly producing better algos and hardware, L2 scaling in crypto can be seen similarly, but nowhere near the same pace of innovation - AI is producing business & consumer value now, there is a sustainable business model, crypto still seems like a separated economic system
- martinald 3y ago
  I agree. This reminds me of ~20 years ago when bandwidth usage was growing incredibly exponentially and everyone thought people in 2024 would be consuming 20TB/day. In reality bandwidth growth slowed down massively, to the point now where it is growing at something like 10-50%/yr and everyone went under because there was far too much capacity. There is a finite amount of fabs that can produce GPUs which sets an upper limit on the production. I doubt we are going to see _that_ many more fabs being built. I also think everyone is hoarding as many GPUs as possible, which makes sense for meta/openai/google but probably much less sense for other players with time - nevermind random corporates that are just jumping on the bandwagon. I really think we'll see a small number of players (more than just openai, meta and google) produce most of the foundational models, then everyone will finetune and infer off them (which are many orders of magnitude less). That's not to say there won't be huge demand for GPUs, but I don't think it's going to be that every fortune 500 needs a 10k GPU cluster. The key differentiator IMO with crypto is crypto by its nature has exponentially more computer resource requirement built into (most) of it. I don't know if AI does after a certain point, and I think there are enormous efficiency gains happening which simply doesn't happen in crypto (which you point out).
  - mistrial9 3y ago
    desktop computer use declined in the same time period in most places. The bandwidth expansion for consumers has been on mobile, and secondly specifically for movie viewing. Also consider a long, deliberate and concerted effort by ISPs across the USA to limit bandwidth for commercial reasons e.g. lying about max download speeds repeatedly and on the record, while rate limiting.
jszymborski 3y ago
Inference and fine-tuning are only going to get more efficient. Training likewise. Compare this to crypto where it increases with time. Im not too worried about this. Maybe we'll have accelerators for inference, but only if they can be cheap and efficient alternatives to GPUs (again, for inference).

Gormo 3y ago

It's odd to characterize that as a "risk". It seems to follow that increases in demand for computing use cases would in turn generate more demand for the hardware required to implement them.

INTPenis 3y ago
The article was about electricity consumption, so in the context of environmental impact, or consumer market impact, it is a risk.
- Gormo 3y ago
  Consumption of electricity has negligible environmental impact. The relevant discussion here would pertain to considerations around generating power, regardless of what downstream use cases the power is being applied to. Pretty much all activity in modern society is going to consume electricity, and overall demand is not going to be decreasing in the first place, so it seems a bit silly to look at this from the demand side: we're always going to need more and more power, and the focus is properly on how to generate power in a clean and scalable way regardless of what it's being used for. Consumer market impact is an interesting topic, though, if there are massive spikes in demand that could drive prices up for other users of electricity. It will be important to ensure to minimize artificial impediments to the expansion of the supply side to mitigate that risk.

verdverm 3y ago

More data & compute still produces noticeably better results, and probably always will (?) The difference is that the AI field is - rapidly producing better algos and hardware, L2 scaling in crypto can be seen similarly, but nowhere near the same pace of innovation - AI is producing business & consumer value now, there is a sustainable business model, crypto still seems like a separated economic system

martinald 3y ago
I agree. This reminds me of ~20 years ago when bandwidth usage was growing incredibly exponentially and everyone thought people in 2024 would be consuming 20TB/day. In reality bandwidth growth slowed down massively, to the point now where it is growing at something like 10-50%/yr and everyone went under because there was far too much capacity. There is a finite amount of fabs that can produce GPUs which sets an upper limit on the production. I doubt we are going to see _that_ many more fabs being built. I also think everyone is hoarding as many GPUs as possible, which makes sense for meta/openai/google but probably much less sense for other players with time - nevermind random corporates that are just jumping on the bandwagon. I really think we'll see a small number of players (more than just openai, meta and google) produce most of the foundational models, then everyone will finetune and infer off them (which are many orders of magnitude less). That's not to say there won't be huge demand for GPUs, but I don't think it's going to be that every fortune 500 needs a 10k GPU cluster. The key differentiator IMO with crypto is crypto by its nature has exponentially more computer resource requirement built into (most) of it. I don't know if AI does after a certain point, and I think there are enormous efficiency gains happening which simply doesn't happen in crypto (which you point out).
- mistrial9 3y ago
  desktop computer use declined in the same time period in most places. The bandwidth expansion for consumers has been on mobile, and secondly specifically for movie viewing. Also consider a long, deliberate and concerted effort by ISPs across the USA to limit bandwidth for commercial reasons e.g. lying about max download speeds repeatedly and on the record, while rate limiting.

jszymborski 3y ago

Inference and fine-tuning are only going to get more efficient. Training likewise. Compare this to crypto where it increases with time. Im not too worried about this. Maybe we'll have accelerators for inference, but only if they can be cheap and efficient alternatives to GPUs (again, for inference).