5 ms·
Google uncloaks once-secret server
- intregus 18y agoThis seems real despite the post date.
- durana 18y agoI've been in data centers hosting Google systems. This picture looks very similar if not identical to what I've seen, so I'd say it is most definitely real.
- stanley 18y agoCould you verify if Google still uses Hitachi Deskstar drives? I recall reading an article about the testing Google conducted to find the most reliable drives, but they refused to publish the findings.
- pert 18y agoThey did actually publish their results. It's an interesting read: http://research.google.com/archive/disk_failures.pdf http://research.google.com/archive/disk_failures.pdf
- ciupicri 18y agoThat paper does not mention any specific brands or models.
- durana 18y agoI wasn't ever close enough to see what kind of disks they were.
- durin42 18y agoBased on their white paper on MTBF of hard disks, I'd guess that they avoid favoring any single vendor to lower the chance of a bad batch of drives all failing at once.
- wmf 18y agoThis server appears to be from 2005; it looks like Google is still keeping some secrets.
- ars 18y agoThe hard disks are dated Nov 2007. The motherboard is from 2005, but it's got decent specs http://www.gigabyte.com.tw/Products/Networking/Products_Spec.aspx?ProductID=1075 http://www.gigabyte.com.tw/Products/Networking/Products_Spec... so they may still be using them. Although I'm surprised they went with Xeon CPU's, and I don't see anything on gigabytes page about 12V only for the board.
- andr 18y agoIt's quite possible that they got Gigabyte to do a custom design for them, given their volumes. Or, the 5v conversion is done in an extra device between the power supply and the mother board power socket (ATX or whatever it is).
- cdibona 18y agoWe did indeed work with gigabyte on that board.
- wmf 18y agoGiven all Google's talk about power efficiency, it's debatable whether power-hogging NetBurst processors are "decent".
- deleted 18y ago[deleted]
- pmjordan 18y agoNote that the publicly available Gigabyte board you link to is the GA-9IVDT, the name on the board in the photo is GA-9IVDP. Moreover, they look nothing like each other. I also can't tell where the 5V rail for the hard disks is coming from: http://i.i.com.com/cnwk.1d/i/bto/20090401/Google_server-1.jpg?tag=mncol;txt http://i.i.com.com/cnwk.1d/i/bto/20090401/Google_server-1.jp... The only wires from/to the PSU are those running to the battery and the 12V and ground wires for the motherboard.
- andr 18y agoI for one am surprised that those are 2U servers and they haven't gone to 1U to double their space efficiency.
- jacquesm 18y agoit looks like two of them may fit side-by-side, like this they get to use the big ram sticks and a cheap psu, as well as fans that will last a lot longer than those pesky 1u fans. They may also use 'deep' racks to the machines have the PSU's in the middle and the fronts on the facing sides of the cabinets, one front, one back. That would make it twice as efficient as 1U, it would also explain why the cables go out the front.
- idm 18y agoI switched back and forth between 1U and 2U setups for the same motherboard, and I could run my system in higher ambient temperature with less airflow when using a 2U heatsink. From the patent-looking picture in the article, it looks like the racks are 1-deep, so I don't think they're using 2-deep racks.
- durana 18y agoI've seen them two deep side by side. They had custom racks that reminded me of those racks of trays of food you'd see in large production kitchens. There were four systems per level/tray.
- iamelgringo 18y agoThe server rack that they have at the Computer History Museum has them stacked 4 systems per tray as well.
- nessence 18y ago1u limits airflow, probably too hot to stack HD that close, and 1u power supplies were probably cost prohibitive.
- ars 18y agoThe article says the battery backup was a secret but: http://www.webhostingtalk.com/archive/index.php/t-492779.html http://www.webhostingtalk.com/archive/index.php/t-492779.htm... and at the time at least, he had linked to a picture of the board (but it's down now).
- pmjordan 18y agoArticle seems to no longer be available from the original link, but it's still up here: http://news.cnet.com/8301-1001_3-10209580-92.html?part=rss&subj=news&tag=2547-1_3-0-20 http://news.cnet.com/8301-1001_3-10209580-92.html?part=rss&#...
- amitt 18y agoHad the read the article carefully to see if i wasn't "getting" the joke. It's legit, and very cool that they were so innovative with their data centers.
- catz 18y agoNo liquid cooling? How efficient is the power supplies?
- hs 18y agoOne Server to rule them all
- oomkiller 18y agoThe battery integration is quite interesting, although I'd bet they are using it for purposes other than "having to buy one big UPS." One benefit I see to it is high availability, since the site UPS going out won't take down the servers, since they all have their own. Plus, I'd bet it makes the guys that work on the power buss sleep sounder at night, since its easier to work on without worrying about taking out the whole datacenter.
- jonah 18y agoSince they're running the PSUs near capacity. the batteries probably provide some additional headroom under heavy load spikes.
- nradov 18y agoYes, there have been a few publicized incidents at large hosting centers where the central UPS either failed or didn't kick in when it was supposed to. By putting the battery in the server it eliminates that single point of failure.
- pert 18y agoAnyone got a link to the "video tour Google presented of its data center containers"?
- wallflower 18y agoIt's a video of the official video but passable (until Google releases the official video shortly): http://www.youtube.com/watch?v=bs3Et540-_s http://www.youtube.com/watch?v=bs3Et540-_s
- ieatpaste 18y agoWhile searching for the actual battery patent, I ran across the modular data center patent: http://www.google.com/patents?id=7TypAAAAEBAJ&printsec=abstract http://www.google.com/patents?id=7TypAAAAEBAJ&printsec=a... and also this power supply research paper abstract: http://research.google.com/pubs/pub32467.html http://research.google.com/pubs/pub32467.html I was also surprised at some of the patents awarded. While the claims were very specific, the ideas were very generic, such as mobile payments via cell phone. Take a look for yourself: http://www.google.com/patents?zoom=4&q=inassignee%3Agoogle&btnG=Search+Patents http://www.google.com/patents?zoom=4&q=inassignee%3Agoog... If anyone finds the battery/server patent, I would be extremely interested.
- wallflower 18y agoNot filed under Google but under Exaflop LLC (one quintillion floating point operations per second) [hat tip to: http://www.datacenterknowledge.com/archives/2008/02/15/google-files-patent-on-ups-architecture/ http://www.datacenterknowledge.com/archives/2008/02/15/googl...] Patent application: 20080030078 "Each UPS includes a battery selectively connectable across a DC bus, and a AC-to-DC rectifier that converts an AC input voltage to a single output voltage on the DC bus. The regulated DC bus voltage may be close to the battery's fully charged voltage..." http://appft1.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d=PG01&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.html&r=1&f=G&l=50&s1=%2220080030078%22.PGNR.&OS=DN/20080030078&RS=DN/20080030078 http://appft1.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Se...
- Semiapies 18y agoPretty much every patent is a specific implementation of a generic idea.
- cool-RR 18y agoI don't understand: How can one battery per server be more efficient than a big, shared battery for all of them?
- enneff 18y agoTo hazard a guess I'd say it's to do with voltage step up/down efficiency.
- pert 18y agoI agree. Standard UPSes convert AC to DC and back, in order to power the server PSUs and these conversions are inefficient. If the battery is in the server (after the PSU), you can miss out these conversions and save a lot of power.
- gamache 18y agoI'd also submit that low-power electronic components are far more readily available (i.e. cheap) than ones which can handle a lot of juice.
- jwilliams 18y agoCould also be to do with management, particularly switching - switching large DC currents is problematic (i.e. it arcs). It's easier to manage on a smaller scale.
- wglb 18y agoWith the one battery per server, you eliminate one moving part. Well not moving, but you leave out the step of going from dc power to ac power. With their setup, you take 220 vac (note the label on the power supply) to the server and to dc. Any conversion step costs heat.
- asciilifeform 18y ago1) No transmission/conversion loss. It is probably soldered directly to the DC power regulator on the mainboard. 2) The servers are frequently removed for maintenance. Thus, the batteries will also be checked/replaced along with the machines they serve. As a bonus, there is always exactly the right amount of battery capacity in the system.
- gstar 18y agoSee how the hard drives are mounted back-to-back? I'm guessing that is designed to cancel the mechanical noise by putting the drives 180deg out of phase from each other, extending the life of the drives. I believe that is patented (by Avid?) and that might be why Google was cagey about releasing specs.
- yardie 18y agoBattery pack is a good idea. I have a UPS that is just about dead (~1 minute of backup, but I keep it for spikes and drops). I thought it would be great if the PSU had a built-in UPS. The space between the PSU and drive bay would be a perfect spot, no need for the heavy switching hardware since the PSU can deliver the 12 or 13.6 volts needed for the battery to charge.
- nradov 18y agoWhat battery chemistry are they using? It looks like a lead-acid package. Those are cheap, safe, and reliable. But they have very low energy density and there are disposal issues.
- vicaya 18y agoI don't see how they can patent battery per motherboard design. Isn't that how laptops work? OTOH, this design makes a lot of sense. One of the biggest failure mode for rack-wise failure I've seen is human error from maintenance crews: trip over or unplug the wrong a power cord (I've seen this happen twice already from different companies.) This design completely shields this type of errors. But, it should not be patentable!
- stcredzero 18y agoThe battery-per motherboard makes eminent sense. Between economies of scale driving down the price of small AGM and gel-cell lead-acid batteries (1) and the increased efficiency of having the battery on board, it's an obvious win! (1) - ironically, this is probably because of retail and commercial sales of Uninterruptible Power Supplies!
- tutwabee 18y agoWas it really necessary to patent the built-in battery design? That doesn't really seem original enough to warrant a patent.
- leej 18y agoI dont think these are current.