6 ms·

The main thing stopping me from going beyond 2x 4090’s in my home lab is power. Anything around ~2k watts on a single circuit breaker is likely to flip it, and

The main thing stopping me from going beyond 2x 4090’s in my home lab is power. Anything around ~2k watts on a single circuit breaker is likely to flip it, and that’s before you get to the costs involved of drawing that much power for multiple days of a training run. How did you navigate that in a (presumably) residential setting?

abound 2y ago
Not OP, but my current home had a dedicated 50A/240V circuit because the previous owner did glass work and had a massive electric kiln. I can't imagine it was cheap to install, but I've used it for beefy, energy hungry servers in the past. Which is all to say its possible in a residential setting, just probably expensive.
- woleium 2y ago
  Yes, or something like a residential aircon heatpump will need a 40a circuit too. Car charging usually has a 30a. Electric oven is usually 40a. There’s lots of stuff that uses that sort of power residentially
throwthrowuknow 2y ago
Not speaking from direct experience building a rig like this but the blog post mentions having 3 power supplies so the most direct solution would be to put each on their own dedicated circuit. As long as you have space in your electrical box this is straightforward to do though I would recommend having an electrician do the wiring if you aren’t experienced with that type of home electrical work.
- gizmo686 2y ago
  Even without space in the existing box, installing a subpanel isn't that much more of a cost.
- XMasterrrr 2y ago
  Actually, putting each PSU on its own circuit is crazy dangerous. In the scenario of your suggestion, if one goes out, you are in for a fire. Highly recommend against that.
  - gizmo686 2y ago
    This might be one of those reasons why I'm not an electritian, but is it dangerous. Back when I worked at a high-availability data center, all of our servers had dual psus, plugged into seperate circuits. The transformer in the PSUs should electrical isolate the mains voltage from the low voltage side, so you aren't going to cause a short across the two circuts. The only risk I see is a cascade failure, where the increased load on the second circuit causes its breaker to trip.
  - throwthrowuknow 2y ago
    Do they not have fuses? Use 15A breakers?
littlestymaar 2y ago
Then juste add a 32A circuit breaker to your electrical installation, it's not a big deal really.
bluedino 2y ago
Take your typical 'GPU node', which would be a Dell/HP/SuperMicro with 4-8 NVIDIA H100's and a single top high level AMD/Intel CPU. You would need 2-4 240v outlets (30A). In the real world you would plug them into a PDU such as: https://www.apc.com/us/en/product/AP9571A/rack-pdu-basic-1u-30a-208v-10-c13/ https://www.apc.com/us/en/product/AP9571A/rack-pdu-basic-1u-... Each GPU will take around 700W and then you have the rest of the system to power, so depending on CPU/RAM/storage... And then you need to cool it!
- XMasterrrr 2y ago
  Yeah, you get it. I have two PDUs with NEMA L6-30P plugin, and two NEMA L6-30R outlet. Each outlet is connected to it's dedicated 30A 240v circuit (which gives you a total of 60A, or 6000w). Cooling is its own story... Man, figuring out this process was hella of a journey
tcdent 2y ago
I can't believe a group of engineers are so afraid of residential power. It is not expensive, nor is it highly technical. It's not like we're factoring in latency and crosstalk... Read a quick howto, cruise into Home Depot and grab some legos off the shelf. Far easier to figure out than executing "hello world" without domain expertise.
- 3eb7988a1663 2y ago
  People can and do die from misuses of electricity. Not a move-fast-and-break things kind of domain.
  - varispeed 2y ago
    You only "break" once...
    - lbotos 2y ago
      I've been learning Japanese and a favorite of mine is: 一体 Which is used as "what the heck" but it's direct kanji translation is one body. https://jisho.org/word/%E4%B8%80%E4%BD%93 https://jisho.org/word/%E4%B8%80%E4%BD%93
      - varispeed 2y ago
        Fun fact as a kid I stuck my fingers in the loose mains wires as we were playing at unfinished building. The wires were live and I still remember it felt like it's going to break my arm. Fortunately I only got a slight burn. This got me interested in electronics which I started studying later in my life.
- gizmo686 2y ago
  A good engineer knows the difference between safe and dangerous. Setting up an AI computer is safe. Maybe you trip a circut. Maybe you interfere with something else running on your hobby computer. But nothing bad can really happen. Residential electrical is dangerous. Maybe you electrocute yourself. Maybe you cause a fire 5 years down the line. Maybe you cause a fire for the next owner because you didn't know to protect the wire with a metal plate so they drill into it. Having said that, 2 4090s will run you aroud $5,000, not counting any of the surrounding system. At that cost point, hireing an electritian would not be that big of an expense relativly speaking. Also, if you are at the point where you need to add a circut for power, you might need to seriously consider cooling, which could potentially be another side quest.
deleted 2y ago
[deleted]
J_Shelby_J 2y ago
I’m running two 3090s on a 700w psu. You definitely can get more than that out of 2000w bus. I wrote a blog on reducing the power limits of nvidia gpus. Definitely try it out. https://shelbyjenkins.github.io/blog/power-limit-nvidia-linux/ https://shelbyjenkins.github.io/blog/power-limit-nvidia-linu...
- smcnally 2y ago
  Thank you for this post. I’d read it in ~June and it helped quite a bit with manual ‘nvidia-smi’ runs. I just recently created the systemd service description and am still delving related power and performance possibilities.
- XMasterrrr 2y ago
  Hey man, I think I came across your blog at some point before while trying to figure out my own power plan for this beast (check my comment to OP for more context), so kudos to you for that. I would say that power limiting is a potential work around, and it should work perfectly fine for inference, but when it comes to trainning you will want to squeeze every ounce of power. So, depends on your goal. What CPU/Mobo/Storage are you running with those two 3090s for a 700w to work? I am gonna say, if at any point you're pushing more than 500w out of that PSU, you might be risking the 80% safety rule. I would have at least used a 850w just to be safe with two 3090s + rest of hardware.
sandos 2y ago
This is funny as a european, since we have many, many groups where we reguarly will run 2kW, and some, loads. Really no issue, but I guess lower voltage makes it a problem.
- sixothree 2y ago
  Yup. We typically have 20 amp breakers in living portions of the house and it's common practice for most devices to top out at 1500 watts. But from your description, you would still need three lines and three breakers. So. I'm not understanding your point.
- XMasterrrr 2y ago
  Most outlets rely on 120v 15amp breakers here in the US, only washer/dryer/kitchenware require the higher end 240v breakers
slavik81 2y ago
Not the OP, but I hired an electrician to put a 30A 240V circuit with a NEMA L6-30R recepticle next to my electrical panel. It was 600 CAD. You can probably get it done cheaper. He had to disconnect another circuit and make a trip to the hardwate store because I told him to bring the wrong breaker.
- XMasterrrr 2y ago
  Yup, that's exactly what I had installed.
orbital-decay 2y ago
>Anything around ~2k watts on a single circuit breaker is likely to flip it I'm curious, how do you use e.g. a washing machine or an electric kettle, if 2kW is enough to flip your breaker? You should simply know your wiring limits. Breaker/wiring at my home won't even notice this.
- immibis 2y ago
  Americans do not have electric kettles and need special circuits for electric clothes dryers.
  - lolinder 2y ago
    We have an electric kettle in the US and it runs just fine drawing 1500W. You're correct that the dryer is on a larger circuit, though.
    - beAbU 2y ago
      > and it runs just fine drawing 1500W. You think that this is "just fine" because you've never experienced the glory that is a 3kW kettle!
      - rootusrootus 2y ago
        I just get 99C water from a tap next to my kitchen sink. Why do people still use kettles?
        deleted 2y ago
        [deleted]
        dymk 2y ago
        Because they don’t have a spare few grand for an instant hot plus installation
        2y ago
teaearlgraycold 2y ago
I’ve ran 3x L40S on a 1650W PSU on a normal 120V 20A circuit.
- XMasterrrr 2y ago
  You've got a much stronger heart than me man
  - teaearlgraycold 2y ago
    Sometimes you just need to YOLO it.
GaggiX 2y ago
I use a hair dryer that is a little bit more than 2kw, but I guess because of the 120V it would be a problem in the US. 16 amps x 120v = 1920W, it would probably trip after several minutes. 16 amps x 230v = 3680W, it wouldn't trip.
- sva_ 2y ago
  When my gf first came to Europe she brought her hairdryer from the US and plugged it in using an adapter that just reroutes the wires. She was unaware of the voltage difference (or thought the adapter would adjust it.) That thing started spewing fire pretty much immediately and I luckily quickly realized what she was doing and was able to pull the plug (I hadn't noticed that she brought her own hair dryer.) Luckily she wasn't pointing it at herself ...
pupdogg 2y ago
You can run a setup of 8x 4090 GPUs using 4x 1200W 240V power supplies (preferably HP HSTNS-PD30 Platinum Series), with a collective use of just around 20-amps, meaning it can easily run on a single 240V 20-amp breaker. This should be easily doable in a home where you typically have a 100 to 200A main power panel. Running 4x 1200W power supplies 24 hours a day will consume 115.2 kWh per day. At an electricity rate of $0.12 per kWh, this will cost approximately $13.82 per day or around $414.72 per month. FYI, I can handle electrical system design and sheet metal enclosure design/fabrication for these rigs, but my software knowledge is limited when it comes to ML. If anyone's interested, I'd love to collaborate on a joint venture to produce these rigs commercially.
- swader999 2y ago
  You'll have helicopters over your house.
  - pupdogg 2y ago
    Believe it or not, a buddy of mine was dealing with crazy high power bills for three months and couldn’t figure out why. He tried everything to cut back, but nothing worked. Finally, he called an electrician, who found a hidden short in a 30-amp circuit that was constantly drawing power without tripping the breaker. After fixing the issue, his bills went back to normal—no helicopters were involved during the process!
    - fennecbutt 2y ago
      And all that heat was going...where?
  - w-ll 2y ago
    Good thing i can see them coming, and happy to offer anyone a cup of coffee/tea to understand im just the biggest nerd in the hoa.
  - defrost 2y ago
    Already have to deal with Robinson R22's during mustering season, a few more won't hurt.
XMasterrrr 2y ago
Oh yeah, my original setup was an RTX 4090 + an RTX 3090, and I swear one night I had the circuit breaker trip more than 15 times before I gave up. I have a UPS so I would run to the box before my system shuts down. Most houses are equipped with 15amp 120v breakers, these should never exceed 1500w, and their max is 1800w but then you're really risking it. So, as mentioned on the article, I actually have installed (2) 30amp 240v breakers dedicated entirely for this setup (and the next one in case I decide to expand to 16x GPUs over 2 nodes lol). Each breaker is supposed to power up to 6000w at ease. I also installed a specific kind of power outlet that can handle that kind of current, and I have done some extreme research into PDUs. I plan on posting about all of that in this series (part 3 according to my current tentative plans) so stay tuned and maybe bookmark the website/add the RSS feed to your digest/or follow me on any of the socials if this is something that you wanna nail down without spending a month on research like me :'D
- mattnewton 2y ago
  Thanks! Yeah the 4090’s are very thirsty if you let them be, I haven’t played enough with throttling their voltage and how that affects perf. Looking forward to your articles.
fennecbutt 2y ago
Is suppose this is an American view. Most places with 240 you can run anything up to 3kW per socket most of the time. But you can also get a sparky and go for a cheap high current socket install on 240 or even pay a bit more to get 3 phase installed, if you have a valid enough use case. Hell most kettles use 3kw. Tho for a big server I'd get it wired dedicated, same way power showers are done (7-12~ kW)

abound 2y ago

Not OP, but my current home had a dedicated 50A/240V circuit because the previous owner did glass work and had a massive electric kiln. I can't imagine it was cheap to install, but I've used it for beefy, energy hungry servers in the past. Which is all to say its possible in a residential setting, just probably expensive.

woleium 2y ago
Yes, or something like a residential aircon heatpump will need a 40a circuit too. Car charging usually has a 30a. Electric oven is usually 40a. There’s lots of stuff that uses that sort of power residentially

throwthrowuknow 2y ago

Not speaking from direct experience building a rig like this but the blog post mentions having 3 power supplies so the most direct solution would be to put each on their own dedicated circuit. As long as you have space in your electrical box this is straightforward to do though I would recommend having an electrician do the wiring if you aren’t experienced with that type of home electrical work.

gizmo686 2y ago
Even without space in the existing box, installing a subpanel isn't that much more of a cost.
XMasterrrr 2y ago
Actually, putting each PSU on its own circuit is crazy dangerous. In the scenario of your suggestion, if one goes out, you are in for a fire. Highly recommend against that.
- gizmo686 2y ago
  This might be one of those reasons why I'm not an electritian, but is it dangerous. Back when I worked at a high-availability data center, all of our servers had dual psus, plugged into seperate circuits. The transformer in the PSUs should electrical isolate the mains voltage from the low voltage side, so you aren't going to cause a short across the two circuts. The only risk I see is a cascade failure, where the increased load on the second circuit causes its breaker to trip.
- throwthrowuknow 2y ago
  Do they not have fuses? Use 15A breakers?

littlestymaar 2y ago

Then juste add a 32A circuit breaker to your electrical installation, it's not a big deal really.

bluedino 2y ago

Take your typical 'GPU node', which would be a Dell/HP/SuperMicro with 4-8 NVIDIA H100's and a single top high level AMD/Intel CPU. You would need 2-4 240v outlets (30A). In the real world you would plug them into a PDU such as: https://www.apc.com/us/en/product/AP9571A/rack-pdu-basic-1u-30a-208v-10-c13/ https://www.apc.com/us/en/product/AP9571A/rack-pdu-basic-1u-... Each GPU will take around 700W and then you have the rest of the system to power, so depending on CPU/RAM/storage... And then you need to cool it!

XMasterrrr 2y ago
Yeah, you get it. I have two PDUs with NEMA L6-30P plugin, and two NEMA L6-30R outlet. Each outlet is connected to it's dedicated 30A 240v circuit (which gives you a total of 60A, or 6000w). Cooling is its own story... Man, figuring out this process was hella of a journey

tcdent 2y ago

I can't believe a group of engineers are so afraid of residential power. It is not expensive, nor is it highly technical. It's not like we're factoring in latency and crosstalk... Read a quick howto, cruise into Home Depot and grab some legos off the shelf. Far easier to figure out than executing "hello world" without domain expertise.

3eb7988a1663 2y ago
People can and do die from misuses of electricity. Not a move-fast-and-break things kind of domain.
- varispeed 2y ago
  You only "break" once...
  - lbotos 2y ago
    I've been learning Japanese and a favorite of mine is: 一体 Which is used as "what the heck" but it's direct kanji translation is one body. https://jisho.org/word/%E4%B8%80%E4%BD%93 https://jisho.org/word/%E4%B8%80%E4%BD%93
    - varispeed 2y ago
      Fun fact as a kid I stuck my fingers in the loose mains wires as we were playing at unfinished building. The wires were live and I still remember it felt like it's going to break my arm. Fortunately I only got a slight burn. This got me interested in electronics which I started studying later in my life.
gizmo686 2y ago
A good engineer knows the difference between safe and dangerous. Setting up an AI computer is safe. Maybe you trip a circut. Maybe you interfere with something else running on your hobby computer. But nothing bad can really happen. Residential electrical is dangerous. Maybe you electrocute yourself. Maybe you cause a fire 5 years down the line. Maybe you cause a fire for the next owner because you didn't know to protect the wire with a metal plate so they drill into it. Having said that, 2 4090s will run you aroud $5,000, not counting any of the surrounding system. At that cost point, hireing an electritian would not be that big of an expense relativly speaking. Also, if you are at the point where you need to add a circut for power, you might need to seriously consider cooling, which could potentially be another side quest.

deleted 2y ago

[deleted]

J_Shelby_J 2y ago

I’m running two 3090s on a 700w psu. You definitely can get more than that out of 2000w bus. I wrote a blog on reducing the power limits of nvidia gpus. Definitely try it out. https://shelbyjenkins.github.io/blog/power-limit-nvidia-linux/ https://shelbyjenkins.github.io/blog/power-limit-nvidia-linu...

smcnally 2y ago
Thank you for this post. I’d read it in ~June and it helped quite a bit with manual ‘nvidia-smi’ runs. I just recently created the systemd service description and am still delving related power and performance possibilities.
XMasterrrr 2y ago
Hey man, I think I came across your blog at some point before while trying to figure out my own power plan for this beast (check my comment to OP for more context), so kudos to you for that. I would say that power limiting is a potential work around, and it should work perfectly fine for inference, but when it comes to trainning you will want to squeeze every ounce of power. So, depends on your goal. What CPU/Mobo/Storage are you running with those two 3090s for a 700w to work? I am gonna say, if at any point you're pushing more than 500w out of that PSU, you might be risking the 80% safety rule. I would have at least used a 850w just to be safe with two 3090s + rest of hardware.

sandos 2y ago

This is funny as a european, since we have many, many groups where we reguarly will run 2kW, and some, loads. Really no issue, but I guess lower voltage makes it a problem.

sixothree 2y ago
Yup. We typically have 20 amp breakers in living portions of the house and it's common practice for most devices to top out at 1500 watts. But from your description, you would still need three lines and three breakers. So. I'm not understanding your point.
XMasterrrr 2y ago
Most outlets rely on 120v 15amp breakers here in the US, only washer/dryer/kitchenware require the higher end 240v breakers

slavik81 2y ago

Not the OP, but I hired an electrician to put a 30A 240V circuit with a NEMA L6-30R recepticle next to my electrical panel. It was 600 CAD. You can probably get it done cheaper. He had to disconnect another circuit and make a trip to the hardwate store because I told him to bring the wrong breaker.

XMasterrrr 2y ago
Yup, that's exactly what I had installed.

orbital-decay 2y ago

>Anything around ~2k watts on a single circuit breaker is likely to flip it I'm curious, how do you use e.g. a washing machine or an electric kettle, if 2kW is enough to flip your breaker? You should simply know your wiring limits. Breaker/wiring at my home won't even notice this.

immibis 2y ago
Americans do not have electric kettles and need special circuits for electric clothes dryers.
- lolinder 2y ago
  We have an electric kettle in the US and it runs just fine drawing 1500W. You're correct that the dryer is on a larger circuit, though.
  - beAbU 2y ago
    > and it runs just fine drawing 1500W. You think that this is "just fine" because you've never experienced the glory that is a 3kW kettle!
    - rootusrootus 2y ago
      I just get 99C water from a tap next to my kitchen sink. Why do people still use kettles?
      - deleted 2y ago
        [deleted]
      - dymk 2y ago
        Because they don’t have a spare few grand for an instant hot plus installation
        2y ago

teaearlgraycold 2y ago

I’ve ran 3x L40S on a 1650W PSU on a normal 120V 20A circuit.

XMasterrrr 2y ago
You've got a much stronger heart than me man
- teaearlgraycold 2y ago
  Sometimes you just need to YOLO it.

GaggiX 2y ago

I use a hair dryer that is a little bit more than 2kw, but I guess because of the 120V it would be a problem in the US. 16 amps x 120v = 1920W, it would probably trip after several minutes. 16 amps x 230v = 3680W, it wouldn't trip.

sva_ 2y ago
When my gf first came to Europe she brought her hairdryer from the US and plugged it in using an adapter that just reroutes the wires. She was unaware of the voltage difference (or thought the adapter would adjust it.) That thing started spewing fire pretty much immediately and I luckily quickly realized what she was doing and was able to pull the plug (I hadn't noticed that she brought her own hair dryer.) Luckily she wasn't pointing it at herself ...

pupdogg 2y ago

You can run a setup of 8x 4090 GPUs using 4x 1200W 240V power supplies (preferably HP HSTNS-PD30 Platinum Series), with a collective use of just around 20-amps, meaning it can easily run on a single 240V 20-amp breaker. This should be easily doable in a home where you typically have a 100 to 200A main power panel. Running 4x 1200W power supplies 24 hours a day will consume 115.2 kWh per day. At an electricity rate of $0.12 per kWh, this will cost approximately $13.82 per day or around $414.72 per month. FYI, I can handle electrical system design and sheet metal enclosure design/fabrication for these rigs, but my software knowledge is limited when it comes to ML. If anyone's interested, I'd love to collaborate on a joint venture to produce these rigs commercially.

swader999 2y ago
You'll have helicopters over your house.
- pupdogg 2y ago
  Believe it or not, a buddy of mine was dealing with crazy high power bills for three months and couldn’t figure out why. He tried everything to cut back, but nothing worked. Finally, he called an electrician, who found a hidden short in a 30-amp circuit that was constantly drawing power without tripping the breaker. After fixing the issue, his bills went back to normal—no helicopters were involved during the process!
  - fennecbutt 2y ago
    And all that heat was going...where?
- w-ll 2y ago
  Good thing i can see them coming, and happy to offer anyone a cup of coffee/tea to understand im just the biggest nerd in the hoa.
- defrost 2y ago
  Already have to deal with Robinson R22's during mustering season, a few more won't hurt.

XMasterrrr 2y ago

Oh yeah, my original setup was an RTX 4090 + an RTX 3090, and I swear one night I had the circuit breaker trip more than 15 times before I gave up. I have a UPS so I would run to the box before my system shuts down. Most houses are equipped with 15amp 120v breakers, these should never exceed 1500w, and their max is 1800w but then you're really risking it. So, as mentioned on the article, I actually have installed (2) 30amp 240v breakers dedicated entirely for this setup (and the next one in case I decide to expand to 16x GPUs over 2 nodes lol). Each breaker is supposed to power up to 6000w at ease. I also installed a specific kind of power outlet that can handle that kind of current, and I have done some extreme research into PDUs. I plan on posting about all of that in this series (part 3 according to my current tentative plans) so stay tuned and maybe bookmark the website/add the RSS feed to your digest/or follow me on any of the socials if this is something that you wanna nail down without spending a month on research like me :'D

mattnewton 2y ago
Thanks! Yeah the 4090’s are very thirsty if you let them be, I haven’t played enough with throttling their voltage and how that affects perf. Looking forward to your articles.

fennecbutt 2y ago

Is suppose this is an American view. Most places with 240 you can run anything up to 3kW per socket most of the time. But you can also get a sparky and go for a cheap high current socket install on 240 or even pay a bit more to get 3 phase installed, if you have a valid enough use case. Hell most kettles use 3kw. Tho for a big server I'd get it wired dedicated, same way power showers are done (7-12~ kW)