3 ms·
You can say it takes 1800-5400 W. Not sure where you are estimating it from.
by YetAnotherNick 10mo ago
You can say it takes 1800-5400 W. Not sure where you are estimating it from.
- KaiserPro 10mo agoDepending on what you're doing its taking up to 8GPUs working in parallel to serve those queries.
- YetAnotherNick 10mo agoYes but then the batch size is in 100s or even 1000s. These GPU doesn't serve just 1 user at a time.