4 ms·
From my estimation each second of gpt eats about 0.5-1.5 watthours
by KaiserPro 10mo ago
From my estimation each second of gpt eats about 0.5-1.5 watthours
- YetAnotherNick 10mo agoYou can say it takes 1800-5400 W. Not sure where you are estimating it from.
- KaiserPro 10mo agoDepending on what you're doing its taking up to 8GPUs working in parallel to serve those queries.
- YetAnotherNick 10mo agoYes but then the batch size is in 100s or even 1000s. These GPU doesn't serve just 1 user at a time.