5 ms·
You sure? In my experiments with multi gpu inference, I couldn't get anywhere close to max theoretical power draw.
by fzzzy 2y ago
You sure? In my experiments with multi gpu inference, I couldn't get anywhere close to max theoretical power draw.
- sabareesh 2y agoWell during training all GPUs were consuming max ~ 450W
- fzzzy 2y agoThanks, good to know. Perhaps it is different for diffusion; with llms, layers are generally split across gpus, meaning inference has to happen on one gpu before the values can be passed between the layer split.
- bb88 2y agoYes! His power supplies are 2x1500 Watt. That puts it at 3KW max which is more than a 20A circuit can provide (2400W). The standard outlet is typically rated at 15 amps or 1800W. And the 15A breaker is on one circuit. You can get 20A circuits but they need to be wired for it, and replacing the breaker won't cut it. Assuming his GPU is ~450W (his number) and power supplies are 80% efficient, well that means he's pulling close to ~2400 watts which is super close to the limit of a 20A circuit. 4 * 450 / 0.80 efficiency = 2250W. That doesn't include the power consumed by the CPU or mother board or other things on that circuit. But a 170W CPU would easily push this over 2400W provided by a 20A circuit.