3 ms·

My hard drives alone pull 200W idle and I don’t have that many. Cooling fans (in servers) pull 150W total most of the time but can be up to 300W. Each 10GbE NIC

by jmb99 1y ago

My hard drives alone pull 200W idle and I don’t have that many. Cooling fans (in servers) pull 150W total most of the time but can be up to 300W. Each 10GbE NIC pulls ~40W. PoE switch gives ~30W to each of my APs, and uses ~80W itself, so there’s another 300W. All of my RAM pulls about 250W (admittedly there’s nearly 2TB of it in the rack, but still). Start adding up CPUs idle/average/max power and the numbers get way bigger. If all he’s pulling is 1kW I’m jealous.

xarope 1y ago
I'm curious, are you getting this with a kill-a-watt (or equivalent), or are you adding up wattage by specifications alone? (I do have an epyc with a bunch of memory and storage, but never bothered doing the math since my UPS claims to be able to run with the average load for 30+mins)
- jmb99 1y ago
  Measured at the UPS, which has 1W resolution. Confirmed its accuracy with an amp clamp on the circuit when running the UPS in pass-through mode. Efficiency is even worse most of the time since it’s a double-conversion (online) UPS that I normally run in full conversion mode - something like 80-85% efficiency overall, but because I fall back to generator power relatively frequently I just leave it there for convenience (in case I’m not home to switch it when the generator’s turned on).
Dylan16807 1y ago
If I had more than 100 watts in idle hard drives, I'd start aggressively figuring out how to let them spin down. Maybe his drives are actually doing something 24 hours a day? But probably not. Networking gear taking that much when it's not busy is really unfortunate. Did IEEE slack on adding effective sleep/downclocking features?
- mafuy 1y ago
  It sounds like the 10GbE is not fiber. With fiber, it would take around 5W only, depending on your needs. It's a bit harder to use, but fiber is really the better option if you want a serious homelab.
throw0101b 1y ago
> Cooling fans (in servers) pull 150W total most of the time but can be up to 300W. Oxide Computer found that going from tiny 20mm fans to 80mm dropped their chassis power usage bigly: they found a rack full of 1U servers had 25% of its power going to the fans (efficiency is to the cube of the radius). * https://www.youtube.com/watch?v=hTJYY_Y1H9Q https://www.youtube.com/watch?v=hTJYY_Y1H9Q
- mdaniel 1y ago
  My mental model is that such a choice is why a data center is deafening: all those tiny fans spinning at hella rpm
kalleboo 1y ago
40W for a 10GbE NIC sounds crazy, my internet router has a dual 10 GbE Intel NIC with one 10GBaseT SFP+ and one 10G DAC and the whole router only draws 24 W (Lenovo ThinkCentre Tiny) My whole “rack” draws about 120W total with: aforementioned router, Synology NAS with 4 drives, 2x10G+4x2.5G PoE switch powering a Ubiquiti AP, 16-port 1G switch, and a PowerBook 540c running AppleTalk routing
- sponaugle 1y ago
  I'm running 100G CWDM4s in my homelab, and those only take about 4w each for the optics, so I suspect with the PCIe card it is less than 10w.
- jmb99 1y ago
  Late response I know, but copper 10GbE nics pull lots of power, especially the old X540 ones.
sponaugle 1y ago
1kW sounds pretty good to me! My homelab is between 4 and 8 kW continuously, depending on what I have running. Cooling that homelab is another 400-1000 watts depending on outside air temp.

by jmb99 1y ago

xarope 1y ago

I'm curious, are you getting this with a kill-a-watt (or equivalent), or are you adding up wattage by specifications alone? (I do have an epyc with a bunch of memory and storage, but never bothered doing the math since my UPS claims to be able to run with the average load for 30+mins)

jmb99 1y ago
Measured at the UPS, which has 1W resolution. Confirmed its accuracy with an amp clamp on the circuit when running the UPS in pass-through mode. Efficiency is even worse most of the time since it’s a double-conversion (online) UPS that I normally run in full conversion mode - something like 80-85% efficiency overall, but because I fall back to generator power relatively frequently I just leave it there for convenience (in case I’m not home to switch it when the generator’s turned on).

Dylan16807 1y ago

If I had more than 100 watts in idle hard drives, I'd start aggressively figuring out how to let them spin down. Maybe his drives are actually doing something 24 hours a day? But probably not. Networking gear taking that much when it's not busy is really unfortunate. Did IEEE slack on adding effective sleep/downclocking features?

mafuy 1y ago
It sounds like the 10GbE is not fiber. With fiber, it would take around 5W only, depending on your needs. It's a bit harder to use, but fiber is really the better option if you want a serious homelab.

throw0101b 1y ago

> Cooling fans (in servers) pull 150W total most of the time but can be up to 300W. Oxide Computer found that going from tiny 20mm fans to 80mm dropped their chassis power usage bigly: they found a rack full of 1U servers had 25% of its power going to the fans (efficiency is to the cube of the radius). * https://www.youtube.com/watch?v=hTJYY_Y1H9Q https://www.youtube.com/watch?v=hTJYY_Y1H9Q

mdaniel 1y ago
My mental model is that such a choice is why a data center is deafening: all those tiny fans spinning at hella rpm

kalleboo 1y ago

40W for a 10GbE NIC sounds crazy, my internet router has a dual 10 GbE Intel NIC with one 10GBaseT SFP+ and one 10G DAC and the whole router only draws 24 W (Lenovo ThinkCentre Tiny) My whole “rack” draws about 120W total with: aforementioned router, Synology NAS with 4 drives, 2x10G+4x2.5G PoE switch powering a Ubiquiti AP, 16-port 1G switch, and a PowerBook 540c running AppleTalk routing

sponaugle 1y ago
I'm running 100G CWDM4s in my homelab, and those only take about 4w each for the optics, so I suspect with the PCIe card it is less than 10w.
jmb99 1y ago
Late response I know, but copper 10GbE nics pull lots of power, especially the old X540 ones.

sponaugle 1y ago

1kW sounds pretty good to me! My homelab is between 4 and 8 kW continuously, depending on what I have running. Cooling that homelab is another 400-1000 watts depending on outside air temp.