3 ms·
Except that the article assumes that the GPUs would be using their max TDP all the time, which is incorrect. GPUs will throttle down to 5-20w (depending on the
by angoragoats 2y ago
Except that the article assumes that the GPUs would be using their max TDP all the time, which is incorrect. GPUs will throttle down to 5-20w (depending on the specific GPU). So your actual power consumption is going to be much, much lower, unless you’re literally using your LLM 24/7.