3 ms·
I don't see why 3 years is the right number there. I'm using a 10 year old model GPU to generate tokens locally, and given the bottlenecks, a commercial model f
by zhivota 4mo ago
I don't see why 3 years is the right number there. I'm using a 10 year old model GPU to generate tokens locally, and given the bottlenecks, a commercial model focused on RAM and transfer speeds should have a longer depreciation curve than 3 years.
- Gigachad 4mo agoBecause these GPUs in data centers chew power and take up space. If in 3 years there is a new model that processes far more tokens with the same power and time the economics quickly say the hardware is cheaper to replace than to continue running. As a hobbiest at home the numbers are different and you can afford to do something inefficient.
- wqaatwt 4mo agoFor some time capital costs seem to be dwarfing the operating costs (or at least energy costs) when it comes to GPUs. Of course that will likely change but it might take quite a while (unless the AI bubble violently pops)
- zhivota 4mo agoI still challenge it. After 3 years an H100 has not depreciated to zero, not even close. Even if you want to replace it, it has considerable residual value.