4 ms·

For the extreme opposite of that, Google famously trolled everyone 10 years ago by announcing that every one of their servers had its own in-chassis 12V battery

For the extreme opposite of that, Google famously trolled everyone 10 years ago by announcing that every one of their servers had its own in-chassis 12V battery: https://www.cnet.com/news/google-uncloaks-once-secret-server-10209580/ https://www.cnet.com/news/google-uncloaks-once-secret-server...

jeffbee 6y ago
Yes but Google also later moved to a 48VDC power architecture with the batteries at the bottom of the rack. See http://apec.dev.itswebs.com/Portals/0/APEC%202017%20Files/Plenary/APEC%20Plenary%20Google.pdf http://apec.dev.itswebs.com/Portals/0/APEC%202017%20Files/Pl... page 6.
- tyingq 6y ago
  That presentation is pretty interesting, including Google inventing their own "Switched Tank" DC-DC converters because the existing ones weren't efficient or reliable enough.
  - jeffbee 6y ago
    There's a whole separate paper on that: https://storage.googleapis.com/pub-tools-public-publication-data/pdf/55746647b67067298c9455ece6d405872c9101c8.pdf https://storage.googleapis.com/pub-tools-public-publication-...
LinuxBender 6y ago
And Facebook had small UPS/ATS units at the end of each row. Not sure if they still do that today it was like that when I walked through their datacenter. They did that for the purpose of power efficiency. They lost far less power by having many smaller units.
- lclarkmichalek 6y ago
  What you saw likely wasn't a UPS, but an RPP > A data center typically spans four rooms, called suites, where racks of servers are arranged in rows. Up to four MSBs provide power to each suite. In turn, each MSB supplies up to four 1.25 MW Switch Boards (SBs). From each SB, power is fed to the 190 KW Reactive Power Panels (RPPs) stationed at the end of each row of racks. https://research.fb.com/wp-content/uploads/2016/11/dynamo_facebook_s_data_center-wide_power_management_system.pdf https://research.fb.com/wp-content/uploads/2016/11/dynamo_fa... The UPS role is taken more by BBUs (Battery Backup Units). https://www.youtube.com/watch?v=KNsposM0sJE https://www.youtube.com/watch?v=KNsposM0sJE has lots of info about them. (I work for FB, on entirely unrelated things)
  - LinuxBender 6y ago
    They specifically called em out as UPS and talked about the efficiency improvements over having a dedicated UPS room. I don't remember the percentage numbers though. It was orders of magnitude less power loss. It could be they ditched all of that hardware by now. This was some time ago. You might ask some of the old timers in the DC. They probably have pictures of the hardware.
dilyevsky 6y ago
It’s true! I wrote software that upgraded firmware on every one of those batteries without frying them up (most of the time). There was a public paper/talk on that few years back
WrtCdEvrydy 6y ago
That was actually true, not an actual April Fool's day.
tinus_hn 6y ago
It sounds rather wasteful to have one device generate mains voltage AC power, only to have another device transform it to lower voltage power and rectify it.
- xvf22 6y ago
  I mean chances are they are doing DC distribution. https://datacenterfrontier.com/google-unveils-48-volt-data-center-rack-joins-open-compute/ https://datacenterfrontier.com/google-unveils-48-volt-data-c...

jeffbee 6y ago

Yes but Google also later moved to a 48VDC power architecture with the batteries at the bottom of the rack. See http://apec.dev.itswebs.com/Portals/0/APEC%202017%20Files/Plenary/APEC%20Plenary%20Google.pdf http://apec.dev.itswebs.com/Portals/0/APEC%202017%20Files/Pl... page 6.

tyingq 6y ago
That presentation is pretty interesting, including Google inventing their own "Switched Tank" DC-DC converters because the existing ones weren't efficient or reliable enough.
- jeffbee 6y ago
  There's a whole separate paper on that: https://storage.googleapis.com/pub-tools-public-publication-data/pdf/55746647b67067298c9455ece6d405872c9101c8.pdf https://storage.googleapis.com/pub-tools-public-publication-...

LinuxBender 6y ago

And Facebook had small UPS/ATS units at the end of each row. Not sure if they still do that today it was like that when I walked through their datacenter. They did that for the purpose of power efficiency. They lost far less power by having many smaller units.

lclarkmichalek 6y ago
What you saw likely wasn't a UPS, but an RPP > A data center typically spans four rooms, called suites, where racks of servers are arranged in rows. Up to four MSBs provide power to each suite. In turn, each MSB supplies up to four 1.25 MW Switch Boards (SBs). From each SB, power is fed to the 190 KW Reactive Power Panels (RPPs) stationed at the end of each row of racks. https://research.fb.com/wp-content/uploads/2016/11/dynamo_facebook_s_data_center-wide_power_management_system.pdf https://research.fb.com/wp-content/uploads/2016/11/dynamo_fa... The UPS role is taken more by BBUs (Battery Backup Units). https://www.youtube.com/watch?v=KNsposM0sJE https://www.youtube.com/watch?v=KNsposM0sJE has lots of info about them. (I work for FB, on entirely unrelated things)
- LinuxBender 6y ago
  They specifically called em out as UPS and talked about the efficiency improvements over having a dedicated UPS room. I don't remember the percentage numbers though. It was orders of magnitude less power loss. It could be they ditched all of that hardware by now. This was some time ago. You might ask some of the old timers in the DC. They probably have pictures of the hardware.

dilyevsky 6y ago

It’s true! I wrote software that upgraded firmware on every one of those batteries without frying them up (most of the time). There was a public paper/talk on that few years back

WrtCdEvrydy 6y ago

That was actually true, not an actual April Fool's day.

tinus_hn 6y ago

It sounds rather wasteful to have one device generate mains voltage AC power, only to have another device transform it to lower voltage power and rectify it.

xvf22 6y ago
I mean chances are they are doing DC distribution. https://datacenterfrontier.com/google-unveils-48-volt-data-center-rack-joins-open-compute/ https://datacenterfrontier.com/google-unveils-48-volt-data-c...