3 ms·
GPU compute price is dropping fast and will continue to do so.
by rapsey 2y ago
GPU compute price is dropping fast and will continue to do so.
- philjohn 2y agoBut is it dropping faster than the needs of the next model that needs to be trained?
- tossandthrow 2y agoShort answer is yes. Also, GPU pricing is hardly relevant. From now on we will see dedicated co-processors on the GPU to handle these things. They will keep on keeping up with the demand until we meet actual physical limits.
- dspillett 2y agoThe cost of GPU time isn't just the cost that you see (buying them initially, paying for service if they are not yours, paying for electricity if they are) but the cost to the environment. Data centre power draws are increasing significantly and the recent explosion in LLM model creation is part of that. Yes, things are getting better per unit (GPUs get more efficient, better yet AI-optimised chipsets are an order more efficient than using GPUs, etc.) but are they getting better per unit of compute faster than the number of compute units being used is increasing ATM?