3 ms·
They don't usually max out for single prompts, but concurrency in the form of multiple users and/or multiple agents will make them draw full power.
by CamperBob2 2mo ago
They don't usually max out for single prompts, but concurrency in the form of multiple users and/or multiple agents will make them draw full power.
- touisteur 2mo agoYes that was the second part of my comment. I've been benchmarking many things AI or not on those boards and while one can sometimes get to 600W I haven't seen more than 10% gain for those last 250W. If someone has a workload that gets more from those watts I'd be interested. For now on anything I run (full-capacity, continuous, batched...) capping at 350W seems better in bang-for-bucks when including energy costs (from the GPU + added cooling).