3 ms·
Because these GPUs in data centers chew power and take up space. If in 3 years there is a new model that processes far more tokens with the same power and time
by Gigachad 4mo ago
Because these GPUs in data centers chew power and take up space. If in 3 years there is a new model that processes far more tokens with the same power and time the economics quickly say the hardware is cheaper to replace than to continue running.
As a hobbiest at home the numbers are different and you can afford to do something inefficient.
- wqaatwt 4mo agoFor some time capital costs seem to be dwarfing the operating costs (or at least energy costs) when it comes to GPUs. Of course that will likely change but it might take quite a while (unless the AI bubble violently pops)
- zhivota 4mo agoI still challenge it. After 3 years an H100 has not depreciated to zero, not even close. Even if you want to replace it, it has considerable residual value.