4 ms·
Can someone check my math on this? Supposedly ChatGPT uses 8x A100's per request. If we say each request takes 5 seconds, and a A100 uses 300watts. (300w * 8)
by thewataccount 3y ago
Can someone check my math on this? Supposedly ChatGPT uses 8x A100's per request. If we say each request takes 5 seconds, and a A100 uses 300watts.
(300w * 8) * 5 = 12,000 watt/sec or 12,000 joules
12,000/3600s = 3.3 watt/hour ?
This is just over the amount of power in a lithium AA battery according to wolframalpha.
- 0xfedbee 3y agoTo me that's still a lot. A whole AA battery for a single response. Maybe I'm just built this way.
- thewataccount 3y agoThey do add up a bit. I'm concerned about a future where we do this for _everything_ because that'll truly be a massive scale unless we get smaller models. If we go off 10M queries a day that's 0.0033KW/H / 24 = 0.0001375/D 0.0001375KW/D * 10M = 1375KW/D or 1.375MW/D (if I did that right)
- arbol 3y agoThis is a crazy amount of power for a single request.