4 ms·

Interesting article. Surprised it does not mention DC use in datacenters: http://gigaom.com/2012/01/13/the-next-big-thing-for-data-centers-dc-power/ http://giga

Interesting article. Surprised it does not mention DC use in datacenters: http://gigaom.com/2012/01/13/the-next-big-thing-for-data-centers-dc-power/ http://gigaom.com/2012/01/13/the-next-big-thing-for-data-cen... AC power supplies on computers are notoriously inefficient and "noisy" (as one datacenter facilities manager described them to me--he was referring to the voltage on the line, not sound.) Many larger datacenters have now switched entirely to DC-powered computers to cut down on energy use. Years ago, when I was running my hosting company, DC power supplies were more expensive, but you could typically make up the difference in a year or so (less than the average 2-3 year lifetime of a typical server.) I suspect costs have come down since then. I seem to recall at least one datacenter in SF running DC directly from the grid, as PG&E used to sell it back then. This article makes it seem like that may no longer be the case. Regardless, many datacenters now sell DC as an option, and will have installed their own converters to supply DC power to customers.

Aloha 13y ago
an AC/DC power supply at best like 70-80% efficient DC-DC converters are upwards of 98% efficient. So you loose 10% in the UPS/PDU Scenario (AC - DC (Float on Batteries) AC) then another 20-25% in the power supply. Meaning to generate 100w inside the PC you need 137.5w input, plus whatever distribution losses you have. If you use HVDC supply with rack based DC-DC Converters. So AC-DC (90% efficient) and DC-DC (98% efficient) You can generate 100w with only 112.2w input. Figure that's for the the whole rack, verses per chassis power supply conversion for the AC model. You can do one 3.3v one 5v and one 12v power supply per rack. You're also saving a huge amount of heat too, heat that you dont need to cycle out thru additional A/C Hardware.
- cnvogel 13y ago
  A power-supply with 70-80% efficiency (at, say, around 50% of the power-rating) would be a very, very crappy one. On the other hand, a DC/DC with 98% would be a very good one. So your comparison is not really fair. See e.g. the book mentioned below. The same book quotes overall inefficiencies from power line to the CPU from as low as 48% for the crappy-PSU + UPS + converters to up to 92% for the Google AC/DC+integrated PSU design. E.g. "The Datacenter as a Computer An Introduction to the Design of Warehouse-Scale Machines, Second Edition, Luiz André, Barroso Jimmy, Clidaras Urs Hölzle" page 52 (PDF page 70) figure 4.2 quotes AC/DC supplies with 79% at worst, but the other two numbers quoted are 89% and 92%. That's more reasonable. http://www.morganclaypool.com/doi/pdf/10.2200/S00516ED2V01Y201306CAC024 http://www.morganclaypool.com/doi/pdf/10.2200/S00516ED2V01Y2...
  - Aloha 13y ago
    I'll admit I cherry picked a little. But the numbers I used are largely borne out on the link you provided. Look at page 52 and 53. I think however larger rack/frame mounted DC/DC converters would likely be even more efficient than per chassis DC/DC converters, but would require almost totally custom hardware.
    - lsc 13y ago
      >I think however larger rack/frame mounted DC/DC converters would likely be even more efficient than per chassis DC/DC converters, but would require almost totally custom hardware. Nope, you can buy off the shelf ATX PSUs that accept 48v DC as input, though it's kinda hard to find. It's easier to find, usually, on the higher-end. For switching gear, though? it's a standard option, as 48v DC is standard for telcos. In fact, most of the higher-end datacenters will sell you 48v DC if you ask for it. It's sort of like having an always-online UPS, as they can just leave the battery-banks in-line; no switching from mains power to UPS required. (and that switch is... quite often what fails when a data center loses power.) Whenever I take folks through my setup at coresite santa clara, they are always drawn to the 48vdc battery bank on the other side of the room. "Hey guys, that's not mine." "No, don't touch that giant copper bus bar." (That's the thing about low voltage; you need serious amounts of copper if you want to move very many watts at all.) It used to be a bigger deal than it is now, as the PSU in your server is dramatically better now than it was even just ten years ago. But yeah, from what I can tell? (and I spend what, 10x rent on data-center power? I'm not saying I'm an expert, but I certainly have an incentive to be right.) The negotiation differences between the cost per watt using different voltages is far greater than the efficiency differences of different voltages. Sales people ruin everything. At many places, 208v cost you twice as much per amp as 120v, so you are better off just buying twice as many 120v amps. (I mean, 208v is more efficient for you and for the datacenter, but it's not 10% more efficient, and you get more than 10% more watts that way.) I was at one place where an amp of 208v was only about 30% more expensive than an amp of 120v. Too bad that free ride ended. The problem is that most salespeople don't really understand any of this... they understand how to sell. And a 10% difference in power cost, well, that's less than the referral bonus they give if you have an agreement to feed them leads. (this is why you see so many small companies claiming to have datacenters across the country. They just have referral programs, and they get upwards of 10% of what the customer ends up paying the real data center, essentially in exchange for some slimy seo.) So really, the negotiation game matters a whole lot more than what is actually the most efficient, until you get to the point where you can build your own data-center.
lostlogin 13y ago
I wonder what is done with the heat produced by the rectification. Presumably when you convert power for 7-10 customers at a time they aren't residential customers and the heat must be quite great. It seems a shame to just radiate that all away.
- m4x 13y ago
  They don't do anything with it, judging by the photos of rectifiers mounted on the wall without any heat exchange or recycling equipment to be seen. It's unlikely that there's much heat to recycle though. If there was, those same photos would show significant heat sinks to help dissipate the energy away from the rectifiers. We don't bother recycling any heat from roadside transformers or larger substations either, and they carry a great deal more energy than the rectifiers powering 7 - 10 lift motors.
  - pkulak 13y ago
    Roadside transformers are AC to AC, which is way more efficient and easy than AC to DC. My car runs on DC, and the quick chargers that provide 60 KW are the size of a large closet and have loud fans to dissipate heat.
    - dfox 13y ago
      Largest source of inefficiency (and heat) in common isolated SMPS designs is output rectification, which has to run on frequencies ranging from tens kHz to units of MHz, thus the diodes there tend to spend relatively large proportion of operating time in various half-open-half-closed states. Active rectifiers somewhat improves overall efficiency for high output currents, but the problem with switching time still remains. On the other hand for line-frequency rectifier times required for rectifier to switch can be ignored and only significant source of heat is voltage drop on rectification diodes (ie. 0.6 - 1.5V), so one watt of heat dissipation per one ampere of current is good ballpark.
- Daniel_Newby 13y ago
  Solid-state rectifiers typically have a forward voltage drop of about 1 V. A 25 kW load at 250 V would draw 100 A. The rectifier power would be 1 V × 100 A = 100 W, for an efficiency of 99.6%. Bulk power rectification is quite efficient.
gaius 13y ago
Sun's "carrier grade" kit all ran on 48V DC, I gather it has been the norm in telcos for a long time.
- bediger4000 13y ago
  In the USA, old landline telephones all ran on (nominally) 48V DC, with the "positive" being ground, and the negative 48V "below ground". Telco central offices all have/had huge glass carboy lead/acid batteries, with giant copper buses running 48V DC all over the buildings. It took me years of asking, but I finally found the right person, who told me that the 48V-below-ground negative meant that dissimilar metal corrosion is drastically reduced in telco "outside plant".
- adestefan 13y ago
  Telecom equipment is actually -48v. It's done that way to reduce galvanic corrosion. The system has been in place since the original Bell system was based on 12v batteries.

Aloha 13y ago

an AC/DC power supply at best like 70-80% efficient DC-DC converters are upwards of 98% efficient. So you loose 10% in the UPS/PDU Scenario (AC - DC (Float on Batteries) AC) then another 20-25% in the power supply. Meaning to generate 100w inside the PC you need 137.5w input, plus whatever distribution losses you have. If you use HVDC supply with rack based DC-DC Converters. So AC-DC (90% efficient) and DC-DC (98% efficient) You can generate 100w with only 112.2w input. Figure that's for the the whole rack, verses per chassis power supply conversion for the AC model. You can do one 3.3v one 5v and one 12v power supply per rack. You're also saving a huge amount of heat too, heat that you dont need to cycle out thru additional A/C Hardware.

cnvogel 13y ago
A power-supply with 70-80% efficiency (at, say, around 50% of the power-rating) would be a very, very crappy one. On the other hand, a DC/DC with 98% would be a very good one. So your comparison is not really fair. See e.g. the book mentioned below. The same book quotes overall inefficiencies from power line to the CPU from as low as 48% for the crappy-PSU + UPS + converters to up to 92% for the Google AC/DC+integrated PSU design. E.g. "The Datacenter as a Computer An Introduction to the Design of Warehouse-Scale Machines, Second Edition, Luiz André, Barroso Jimmy, Clidaras Urs Hölzle" page 52 (PDF page 70) figure 4.2 quotes AC/DC supplies with 79% at worst, but the other two numbers quoted are 89% and 92%. That's more reasonable. http://www.morganclaypool.com/doi/pdf/10.2200/S00516ED2V01Y201306CAC024 http://www.morganclaypool.com/doi/pdf/10.2200/S00516ED2V01Y2...
- Aloha 13y ago
  I'll admit I cherry picked a little. But the numbers I used are largely borne out on the link you provided. Look at page 52 and 53. I think however larger rack/frame mounted DC/DC converters would likely be even more efficient than per chassis DC/DC converters, but would require almost totally custom hardware.
  - lsc 13y ago
    >I think however larger rack/frame mounted DC/DC converters would likely be even more efficient than per chassis DC/DC converters, but would require almost totally custom hardware. Nope, you can buy off the shelf ATX PSUs that accept 48v DC as input, though it's kinda hard to find. It's easier to find, usually, on the higher-end. For switching gear, though? it's a standard option, as 48v DC is standard for telcos. In fact, most of the higher-end datacenters will sell you 48v DC if you ask for it. It's sort of like having an always-online UPS, as they can just leave the battery-banks in-line; no switching from mains power to UPS required. (and that switch is... quite often what fails when a data center loses power.) Whenever I take folks through my setup at coresite santa clara, they are always drawn to the 48vdc battery bank on the other side of the room. "Hey guys, that's not mine." "No, don't touch that giant copper bus bar." (That's the thing about low voltage; you need serious amounts of copper if you want to move very many watts at all.) It used to be a bigger deal than it is now, as the PSU in your server is dramatically better now than it was even just ten years ago. But yeah, from what I can tell? (and I spend what, 10x rent on data-center power? I'm not saying I'm an expert, but I certainly have an incentive to be right.) The negotiation differences between the cost per watt using different voltages is far greater than the efficiency differences of different voltages. Sales people ruin everything. At many places, 208v cost you twice as much per amp as 120v, so you are better off just buying twice as many 120v amps. (I mean, 208v is more efficient for you and for the datacenter, but it's not 10% more efficient, and you get more than 10% more watts that way.) I was at one place where an amp of 208v was only about 30% more expensive than an amp of 120v. Too bad that free ride ended. The problem is that most salespeople don't really understand any of this... they understand how to sell. And a 10% difference in power cost, well, that's less than the referral bonus they give if you have an agreement to feed them leads. (this is why you see so many small companies claiming to have datacenters across the country. They just have referral programs, and they get upwards of 10% of what the customer ends up paying the real data center, essentially in exchange for some slimy seo.) So really, the negotiation game matters a whole lot more than what is actually the most efficient, until you get to the point where you can build your own data-center.

lostlogin 13y ago

I wonder what is done with the heat produced by the rectification. Presumably when you convert power for 7-10 customers at a time they aren't residential customers and the heat must be quite great. It seems a shame to just radiate that all away.

m4x 13y ago
They don't do anything with it, judging by the photos of rectifiers mounted on the wall without any heat exchange or recycling equipment to be seen. It's unlikely that there's much heat to recycle though. If there was, those same photos would show significant heat sinks to help dissipate the energy away from the rectifiers. We don't bother recycling any heat from roadside transformers or larger substations either, and they carry a great deal more energy than the rectifiers powering 7 - 10 lift motors.
- pkulak 13y ago
  Roadside transformers are AC to AC, which is way more efficient and easy than AC to DC. My car runs on DC, and the quick chargers that provide 60 KW are the size of a large closet and have loud fans to dissipate heat.
  - dfox 13y ago
    Largest source of inefficiency (and heat) in common isolated SMPS designs is output rectification, which has to run on frequencies ranging from tens kHz to units of MHz, thus the diodes there tend to spend relatively large proportion of operating time in various half-open-half-closed states. Active rectifiers somewhat improves overall efficiency for high output currents, but the problem with switching time still remains. On the other hand for line-frequency rectifier times required for rectifier to switch can be ignored and only significant source of heat is voltage drop on rectification diodes (ie. 0.6 - 1.5V), so one watt of heat dissipation per one ampere of current is good ballpark.
Daniel_Newby 13y ago
Solid-state rectifiers typically have a forward voltage drop of about 1 V. A 25 kW load at 250 V would draw 100 A. The rectifier power would be 1 V × 100 A = 100 W, for an efficiency of 99.6%. Bulk power rectification is quite efficient.

gaius 13y ago

Sun's "carrier grade" kit all ran on 48V DC, I gather it has been the norm in telcos for a long time.

bediger4000 13y ago
In the USA, old landline telephones all ran on (nominally) 48V DC, with the "positive" being ground, and the negative 48V "below ground". Telco central offices all have/had huge glass carboy lead/acid batteries, with giant copper buses running 48V DC all over the buildings. It took me years of asking, but I finally found the right person, who told me that the 48V-below-ground negative meant that dissimilar metal corrosion is drastically reduced in telco "outside plant".
adestefan 13y ago
Telecom equipment is actually -48v. It's done that way to reduce galvanic corrosion. The system has been in place since the original Bell system was based on 12v batteries.