18 ms·

Google throws open doors to its top-secret data center

270 points by Hurdy 14y ago

mseebach 14y ago
It's a nice piece, but nothing new in it, and most certainly no doors were thrown anywhere.
- 3825 14y ago
  Thank you from sparing me from reading the article. :)
- ludovicurbain 14y ago
  +1, I saw this one coming... top secret DC with open doors ? duh.
- lawdawg 14y ago
  True, buts its a good consolidation/confirmation of what public information there is known about Google's datacenters.
- Hurdy 14y ago
  Well, it's the first time that there are photos from inside. There's a bigger gallery here: http://www.google.com/about/datacenters/gallery/ http://www.google.com/about/datacenters/gallery/
  - bruceboughton 14y ago
    That link seems to throw Chrome 15 into an infinite redirect loop (without cutting off after n redirects). Maybe they don't want you to see inside.
    - Hurdy 14y ago
      I'm really curious: What's the reason that you are still using Chrome 15?
      - bruceboughton 14y ago
        Corporate firewall blocks auto-update channel.
        eco 14y ago
        That's pretty amusing that the corporate firewall is making your system far less secure. There has most likely been hundreds of security issues fixed since Chrome 15.
ims 14y ago
Single page version: http://www.wired.com/wiredenterprise/2012/10/ff-inside-google-data-center/all/ http://www.wired.com/wiredenterprise/2012/10/ff-inside-googl...
fourspace 14y ago
I had the pleasure of helping to build and manage these facilities, both hardware and software, for 5 years. It's nice to see some of Google's real innovations reach the public eye. Some of the smartest folks I ever worked with at the company build absolutely mind blowing tech that the outside never has the opportunity to see or appreciate. In fact, while much of the content in the article has been written about before, it's still probably 2-3 years or more behind where Google is actually at. I left in 2010 and did't read about anything I had not experienced.
- j45 14y ago
  Reminds me when msn search spent a billion dollars (or whatever was reported) saying we have more pages than google. Google simply updated the number of pages indexed after Microsoft was done huffing and puffing. It was pretty funny at the time but the lesson wasn't lost on me: With competition, get ahead, stay ahead, and have things already done and implemented so you can announce big accomplishments when it's strategic for you.
  - wpietri 14y ago
    Yes. You can get to the top by competing with others; you stay on top by competing with yourself. There's a great graph in "Toyota Kata" that shows per-worker productivity of major car companies for the last several decades. They all rise together for the early part of the graph. In the 60s, the American car companies level off; Toyota keeps growing. They focused on continuous improvement, while American car companies floundered. The really interesting part of this to me is that it's rooted in a philosophical difference. Toyota was started and run by engineers. The American car companies gave birth to the MBA approach to business. Engineers naturally seek improvement; MBAs seek profit. Google is one of the few major companies with a philosophical background like Toyota's. It's run by nerds. Their goal isn't to increase shareholder value; it's to build great stuff and organize the world's information. Like Toyota, by following their vision, they have generated vast profits and dominated their industry.
    - seivan 14y ago
VikingCoder 14y ago
http://www.google.com/about/datacenters/inside/streetview/ http://www.google.com/about/datacenters/inside/streetview/
- knowaveragejoe 14y ago
  Amusing easter eggs throughout. One guy has rick roll videos playing on both his computers.
  - svdad 14y ago
    Love the storm trooper.
    - dredmorbius 14y ago
      Not very effective security considering the R2 unit rolling past him.
      - jrockway 14y ago
        That wasn't the droid he was looking for.
- squarecat 14y ago
  So, basically, the Lenoir data center is their sightseeing location for "factory tours". The only data stored there is probably Doodles...
  - Evbn 14y ago
    Yeah so you can't actually see the data when you visit a data center.
    - squarecat 14y ago
      Understood, which is why Google, if it's going to all the effort of setting up open door press days, is missing a huge opportunity by not employing hypervisualized metaphors/exhibits to demonstrate how truly amazing a modern data center is, not to mention reinforcing how critical it is for so much of what the average user does every day. They should probably contract a couple Disney Imagineers and do it right. They could benefit from the the humanizing effects and even create a quirky, niche destination in the process. Hell, throw in an "Android Experience" showroom and it'd probably even have legitimate commercial value.
DanBC 14y ago
It's a shame that heat is just dumped outside most of the time. EDIT: The article talks about Google's impressive technical achievements. But there's a lot of energy that's wasted in industry. I don't mean "used inefficiently" (although that's bad too); I mean actually wasted. I used to work at a tiny electronic sub-contracting factory. The morning shift would arrive, turn on the air compressor (2 KW), the reflow ovens (10 KW and 12 KW); and the other machines (about 7 KW). But they'd do that even if the machines were not going to be running. All these KW were being used for no reason at all. And the machines are pretty inefficient anyway. (One of the owners thought powered machines looked more impressive. Energy costs were included in the rent so there was no incentive to think about when the machines were on or off. ) Counting that waste across all the tiny factories in the world, and including all the waste in offices - it's quite a lot.
- jbri 14y ago
  Where else would that heat end up? Fundamentally, all cooling is the process of transferring heat from one place to another.
  - skrause 14y ago
    The heat could be used to heat up houses or provide warm water: http://en.wikipedia.org/wiki/District_heating http://en.wikipedia.org/wiki/District_heating
    - Evbn 14y ago
      Google does this where possible.
    - packetslave 14y ago
      The problem is getting the heat from where it's generated to where it needs to go with any kind of efficiency.
- freyfogle 14y ago
  Agree. Here in London there was big announcement that heat from telehouse would go to homes: http://www.datacenterknowledge.com/archives/2009/04/15/telehouse-to-heat-homes-at-docklands/ http://www.datacenterknowledge.com/archives/2009/04/15/teleh... but searching briefly just now I can't find any followup beyond the announcement, not clear if it became reality.
wilfra 14y ago
Good read but most of that is not new information. I read a lot of that in a book about Google over two years ago. The last ~ 1 page was new though.
- waqf 14y ago
  Officially announcing things that "everybody knows" already can still make a difference. It means that you can ask Google executives about those things in public appearances and they'll at least acknowledge the question even if they refuse to answer substantially.
Tipzntrix 14y ago
They have a team causing water leaks and stealing hardware to test their disaster recovery. That is some serious penetration testing.
- riobard 14y ago
  Isn't that just a simulation, not for real?
  - Tipzntrix 14y ago
    From the tone of the article, it doesn't seem so. They did say the protest was placated with imaginary pizza though. Hmm, like that would work :P
seiji 14y ago
Google should one-up Amazon and get into the Datacenter As A Service market. Service segments: normal cages (I'd rather lease cages from Colorful Pipes, Inc than Equinix), pay-n-go turnkey same-hardware in 3 georedundant locations, and lease-by-rack in multiples of 10 pre-populated racks (racks specified as compute-only or storage-only with 10G interconnects between racks).
- cracell 14y ago
  It's doubtful that they could directly compete as is. Amazon has done well with their services because they eat their own dog food. From everything I read Bezos basically forced them to build this system and consume it for Amazon's own needs. Google has never taken this approach with their APIs and the difference shows very clearly when you consume these products.
  - sses 14y ago
    I assume you're referencing stevey's rant, and if so you're conflating two issues. Steve was talking about the use of APIs between services at the application level, not the API for the datacenter/cloud as a platform.
    - seiji 14y ago
      I think cr is talking about how you get "baby bigtable" and "baby cloudscale" that are copies of internal services, but not what core platforms use themselves.
- erikpukinskis 14y ago
  EC2 is relevant today, but it will become less and less relevant as the Platform-as-a-Service services (S3, SimpleDB, App Engine, Heroku, etc) get better. There will reach a point where fewer and fewer companies will actually use VMs directly. Seems like a reasonable play for Google to sit this round out and focus on winning the next one.
sounds 14y ago
Single page article: (note: HN guidelines suggest always submitting the single-page article) http://www.wired.com/wiredenterprise/2012/10/ff-inside-google-data-center/all/ http://www.wired.com/wiredenterprise/2012/10/ff-inside-googl...
- ImprovedSilence 14y ago
  There should be a guide for hacks on how to view articles on a single page. Yeah, wired is easy, there's a link, but often times it calls for a little php knowledge, or viewing in "print" mode. Or something random. Not always complexly obvious, but a list of tricks to try might be handy...
  - missing_cipher 14y ago
    There are lots of solutions, one of them is: http://www.printwhatyoulike.com/pagezipper http://www.printwhatyoulike.com/pagezipper
  - dailo10 14y ago
    Have you tried the "Auto Pager" Chrome extension? It's pretty handy...
  - heyitsnick 14y ago
    Readability does a very good job of single-paging.
  - xtrahotsauce 14y ago
    I love AutoPatchWork (for chrome) for this purpose: https://chrome.google.com/webstore/detail/autopatchwork/aeolcjbaammbkgaiagooljfdepnjmkfd https://chrome.google.com/webstore/detail/autopatchwork/aeol...
francov88 14y ago
Really cool article - would be amazing to walk through that facility.... love the Google coloured pipes from the pictures
Fando 14y ago
An incredible article!
bhauer 14y ago
All caveats about chrome-dev aside, I find it amusing that this site's navigation does not work in Chrome v24.0.1297.0. Had to use Aurora to view it. Maybe Google really is Sun v2 ("We are the dot in dot-com" == "Where the Internet lives").
Loic 14y ago
I start to be annoyed with the "a power efficiency of 2 is the standard in datacenters". My servers are hosted in a datacenter with a global efficiency of 1.15, proved after more than a year in operation. Announcing that Google is doing 1.2 is simply announcing something wrong and I suppose Google is very happy with this number being provided to the press. It means that some competitor will use it as "Google is the best, they do 1.2, we are at 1.3 we are not too bad", where I bet Google is now near 1.1 or less (they operate without cooling in Belgium for example).
- wmf 14y ago
  PUE 2 probably was the standard when Google started building their DCs. There's a wide range between enterprisey DCs that consider going from 2 to 1.8 an epic win and clouds/hosters who are getting below 1.3. Google's PUE data since 2008 is published at http://www.google.com/about/datacenters/efficiency/internal/index.html http://www.google.com/about/datacenters/efficiency/internal/...
- rryan 14y ago
  http://www.google.com/about/datacenters/efficiency/internal/index.html http://www.google.com/about/datacenters/efficiency/internal/... You're right -- those PUE numbers from the article were talking about their PUE at the time. Google's 2012 average PUE across all facilities was 1.12/1.13 with a minimum PUE of 1.09/1.10. Also, Google puts enormous care into the process of calculating PUE since it's kind of black art and if you aren't careful you'll leave out some aspect of your operation that will mislead you into thinking your PUE is lower than it is.
brown9-2 14y ago
In regards to the disaster testing: How did Google do this time? Pretty well. Despite the outages in the corporate network, executive chair Eric Schmidt was able to run a scheduled global all-hands meeting. The imaginary demonstrators were placated by imaginary pizza. How does one decide what will placate imaginary demonstrators? Who calls them off?
- ubercore 14y ago
  For the purposes of tests like that, they probably just wanted to see that "reasonable action was taken", which will (hand-waving) probably take care of most instances of that type. In the event of real demonstrators, it would just be the opening salvo of damage control, but it's too hard to predicate how a crowd of angry people would react past the first move.
rpearl 14y ago
There are some photos, such as https://www.google.com/about/datacenters/gallery/images/_3000/IDI_018.jpg https://www.google.com/about/datacenters/gallery/images/_300... I wonder why they've mirrored the image (the left side is quite clearly the right side flipped--take a look at the machine identifier labels). What's being hidden?
- cbr 14y ago
  Perfect symmetry makes it more impressive?
- reledi 14y ago
  The blue LEDs in the picture you linked to, indicate that the servers are running smoothly. [1] It's possible that some servers were faulty at the time the picture was taken. More likely, it's to make the image look perfect. [1] http://www.google.com/about/datacenters/gallery/#/all/12 http://www.google.com/about/datacenters/gallery/#/all/12
javajosh 14y ago
Hi Google Platform people. Very nice work. As you may know, Randall Monroe (of xkcd fame) has recently started a feature called "what if" on their site. I would like to post a question to you along those lines: What if Google was tasked with building an orbiting datacenter? How about a Dyson ring, or sphere? How would you do it? If we were to use all matter in the solar system for commodity linux hardware, how much gmail storage would I get? How many flops? And what sorts of computation could you do on this monster? Please answer! This should be fun...
- alxv 14y ago
  Space would be a terrible environment for building a datacenter. The main goals of a datacenter is to make computation as cheap, as fast and as reliable as possible. Having the datacenter orbit around Earth would not help us accomplish any of these goals. First off, building a datacenter in space would not be cheap. It costs around $25,000 to send a kilogram of equipment into a geostationary orbit. [1] So let assume we were to use Dell PowerEdge C1100. Each server costs $14,000 and weights 18kg. [2] This means for each server sent in orbit, you could buy 32 extra ones on Earth. Then, there is the issue of cooling. Although outer space is really cold, its vacuum prevents the heat generated by the machines from being dissipated quickly. Controlling the temperature of a such datacenter would be a very interesting engineering challenge. And then how would you power this datacenter? Converting the excess heat back into electricity could be an interesting option. But most likely, it would need a lot of solar panels. This would make the datacenter cheap to run once built, but the upfront costs would be enormous. And we haven't talked about speed and reliability yet. Since the signal would need to travel about 35,000 km from the geostationary orbit to reach us, communications between Earth and the datacenter would have significant delays. Even at the speed of light, the minimum round trip time would be about 250 milliseconds if we ignore all other possible sources of delay. The hostile space weather would also make it pretty hard to run servers reliability. Radiations would destroy electronics, caused bit to flip randomly and do all bunch of fun stuff to the equipment. But... anyhow! Let's assume anyway that by some magical work of science and Google engineering, we figure ways to manufacture a datacenter directly space for almost nothing by mining the Moon, discover some amazing thermoelectric generators with near 100% efficiency and space shields that blocks almost all radiations. So back our previous example, a high performance PowerEdge gives us up to about 300 GFLOPS of computing power, 192 GB of RAM and 12 TB of storage. Now if we were to convert the total mass of the Moon (7.34767309 × 10²² kg) into one monstrous datacenter, this would give us about 4.0 × 10²¹ servers. It would gives us a whooping 1.2 billions YottaFLOPS (or put differently, 1.2 × 10³³ FLOPS) of compute madness, 0.8 billions YottaBytes and 49 billions YottaBytes of storage. This monster would consume about the equivalent of 1% of the Sun's total power output. [1]: http://www.futron.com/upload/wysiwyg/Resources/Whitepapers/Space_Transportation_Costs_Trends_0902.pdf http://www.futron.com/upload/wysiwyg/Resources/Whitepapers/S... [2]: http://www.dell.com/us/enterprise/p/poweredge-c1100/pd#TechSpec http://www.dell.com/us/enterprise/p/poweredge-c1100/pd#TechS...
dredmorbius 14y ago
When they say that supercomputing is essentially a plumbing problem ... looking at these photos, no kidding.
stock_toaster 14y ago
It is unfortunate (for the rest of us) that datacenter tech is such a competitive advantage for Google. If they were able to share their breakthroughs more readily with others, imagine how much less of the "1.5% of all power globally" datacenters could be using.
no_script 14y ago
Seriously. I can't believe they require JavaScript to view this all this eye-candy and server porn. http://www.google.com/about/datacenters/gallery/ http://www.google.com/about/datacenters/gallery/ I thought GWT was designed to "compile" rendered pages for a wide variety of browsers and permutations of configurations? The pictures are very pretty, but that's really awful of them to release a PR site like this, and force users into using JavaScript. Unforgivable.

270 points by Hurdy 14y ago

mseebach 14y ago

It's a nice piece, but nothing new in it, and most certainly no doors were thrown anywhere.

3825 14y ago
Thank you from sparing me from reading the article. :)
ludovicurbain 14y ago
+1, I saw this one coming... top secret DC with open doors ? duh.
lawdawg 14y ago
True, buts its a good consolidation/confirmation of what public information there is known about Google's datacenters.
Hurdy 14y ago
Well, it's the first time that there are photos from inside. There's a bigger gallery here: http://www.google.com/about/datacenters/gallery/ http://www.google.com/about/datacenters/gallery/
- bruceboughton 14y ago
  That link seems to throw Chrome 15 into an infinite redirect loop (without cutting off after n redirects). Maybe they don't want you to see inside.
  - Hurdy 14y ago
    I'm really curious: What's the reason that you are still using Chrome 15?
    - bruceboughton 14y ago
      Corporate firewall blocks auto-update channel.
      - eco 14y ago
        That's pretty amusing that the corporate firewall is making your system far less secure. There has most likely been hundreds of security issues fixed since Chrome 15.

ims 14y ago

Single page version: http://www.wired.com/wiredenterprise/2012/10/ff-inside-google-data-center/all/ http://www.wired.com/wiredenterprise/2012/10/ff-inside-googl...

fourspace 14y ago

I had the pleasure of helping to build and manage these facilities, both hardware and software, for 5 years. It's nice to see some of Google's real innovations reach the public eye. Some of the smartest folks I ever worked with at the company build absolutely mind blowing tech that the outside never has the opportunity to see or appreciate. In fact, while much of the content in the article has been written about before, it's still probably 2-3 years or more behind where Google is actually at. I left in 2010 and did't read about anything I had not experienced.

j45 14y ago
Reminds me when msn search spent a billion dollars (or whatever was reported) saying we have more pages than google. Google simply updated the number of pages indexed after Microsoft was done huffing and puffing. It was pretty funny at the time but the lesson wasn't lost on me: With competition, get ahead, stay ahead, and have things already done and implemented so you can announce big accomplishments when it's strategic for you.
- wpietri 14y ago
  Yes. You can get to the top by competing with others; you stay on top by competing with yourself. There's a great graph in "Toyota Kata" that shows per-worker productivity of major car companies for the last several decades. They all rise together for the early part of the graph. In the 60s, the American car companies level off; Toyota keeps growing. They focused on continuous improvement, while American car companies floundered. The really interesting part of this to me is that it's rooted in a philosophical difference. Toyota was started and run by engineers. The American car companies gave birth to the MBA approach to business. Engineers naturally seek improvement; MBAs seek profit. Google is one of the few major companies with a philosophical background like Toyota's. It's run by nerds. Their goal isn't to increase shareholder value; it's to build great stuff and organize the world's information. Like Toyota, by following their vision, they have generated vast profits and dominated their industry.
  - seivan 14y ago

VikingCoder 14y ago

http://www.google.com/about/datacenters/inside/streetview/ http://www.google.com/about/datacenters/inside/streetview/

knowaveragejoe 14y ago
Amusing easter eggs throughout. One guy has rick roll videos playing on both his computers.
- svdad 14y ago
  Love the storm trooper.
  - dredmorbius 14y ago
    Not very effective security considering the R2 unit rolling past him.
    - jrockway 14y ago
      That wasn't the droid he was looking for.
squarecat 14y ago
So, basically, the Lenoir data center is their sightseeing location for "factory tours". The only data stored there is probably Doodles...
- Evbn 14y ago
  Yeah so you can't actually see the data when you visit a data center.
  - squarecat 14y ago
    Understood, which is why Google, if it's going to all the effort of setting up open door press days, is missing a huge opportunity by not employing hypervisualized metaphors/exhibits to demonstrate how truly amazing a modern data center is, not to mention reinforcing how critical it is for so much of what the average user does every day. They should probably contract a couple Disney Imagineers and do it right. They could benefit from the the humanizing effects and even create a quirky, niche destination in the process. Hell, throw in an "Android Experience" showroom and it'd probably even have legitimate commercial value.

DanBC 14y ago

It's a shame that heat is just dumped outside most of the time. EDIT: The article talks about Google's impressive technical achievements. But there's a lot of energy that's wasted in industry. I don't mean "used inefficiently" (although that's bad too); I mean actually wasted. I used to work at a tiny electronic sub-contracting factory. The morning shift would arrive, turn on the air compressor (2 KW), the reflow ovens (10 KW and 12 KW); and the other machines (about 7 KW). But they'd do that even if the machines were not going to be running. All these KW were being used for no reason at all. And the machines are pretty inefficient anyway. (One of the owners thought powered machines looked more impressive. Energy costs were included in the rent so there was no incentive to think about when the machines were on or off. ) Counting that waste across all the tiny factories in the world, and including all the waste in offices - it's quite a lot.

jbri 14y ago
Where else would that heat end up? Fundamentally, all cooling is the process of transferring heat from one place to another.
- skrause 14y ago
  The heat could be used to heat up houses or provide warm water: http://en.wikipedia.org/wiki/District_heating http://en.wikipedia.org/wiki/District_heating
  - Evbn 14y ago
    Google does this where possible.
  - packetslave 14y ago
    The problem is getting the heat from where it's generated to where it needs to go with any kind of efficiency.
freyfogle 14y ago
Agree. Here in London there was big announcement that heat from telehouse would go to homes: http://www.datacenterknowledge.com/archives/2009/04/15/telehouse-to-heat-homes-at-docklands/ http://www.datacenterknowledge.com/archives/2009/04/15/teleh... but searching briefly just now I can't find any followup beyond the announcement, not clear if it became reality.

wilfra 14y ago

Good read but most of that is not new information. I read a lot of that in a book about Google over two years ago. The last ~ 1 page was new though.

waqf 14y ago
Officially announcing things that "everybody knows" already can still make a difference. It means that you can ask Google executives about those things in public appearances and they'll at least acknowledge the question even if they refuse to answer substantially.

Tipzntrix 14y ago

They have a team causing water leaks and stealing hardware to test their disaster recovery. That is some serious penetration testing.

riobard 14y ago
Isn't that just a simulation, not for real?
- Tipzntrix 14y ago
  From the tone of the article, it doesn't seem so. They did say the protest was placated with imaginary pizza though. Hmm, like that would work :P

seiji 14y ago

Google should one-up Amazon and get into the Datacenter As A Service market. Service segments: normal cages (I'd rather lease cages from Colorful Pipes, Inc than Equinix), pay-n-go turnkey same-hardware in 3 georedundant locations, and lease-by-rack in multiples of 10 pre-populated racks (racks specified as compute-only or storage-only with 10G interconnects between racks).

cracell 14y ago
It's doubtful that they could directly compete as is. Amazon has done well with their services because they eat their own dog food. From everything I read Bezos basically forced them to build this system and consume it for Amazon's own needs. Google has never taken this approach with their APIs and the difference shows very clearly when you consume these products.
- sses 14y ago
  I assume you're referencing stevey's rant, and if so you're conflating two issues. Steve was talking about the use of APIs between services at the application level, not the API for the datacenter/cloud as a platform.
  - seiji 14y ago
    I think cr is talking about how you get "baby bigtable" and "baby cloudscale" that are copies of internal services, but not what core platforms use themselves.
erikpukinskis 14y ago
EC2 is relevant today, but it will become less and less relevant as the Platform-as-a-Service services (S3, SimpleDB, App Engine, Heroku, etc) get better. There will reach a point where fewer and fewer companies will actually use VMs directly. Seems like a reasonable play for Google to sit this round out and focus on winning the next one.

sounds 14y ago

Single page article: (note: HN guidelines suggest always submitting the single-page article) http://www.wired.com/wiredenterprise/2012/10/ff-inside-google-data-center/all/ http://www.wired.com/wiredenterprise/2012/10/ff-inside-googl...

ImprovedSilence 14y ago
There should be a guide for hacks on how to view articles on a single page. Yeah, wired is easy, there's a link, but often times it calls for a little php knowledge, or viewing in "print" mode. Or something random. Not always complexly obvious, but a list of tricks to try might be handy...
- missing_cipher 14y ago
  There are lots of solutions, one of them is: http://www.printwhatyoulike.com/pagezipper http://www.printwhatyoulike.com/pagezipper
- dailo10 14y ago
  Have you tried the "Auto Pager" Chrome extension? It's pretty handy...
- heyitsnick 14y ago
  Readability does a very good job of single-paging.
- xtrahotsauce 14y ago
  I love AutoPatchWork (for chrome) for this purpose: https://chrome.google.com/webstore/detail/autopatchwork/aeolcjbaammbkgaiagooljfdepnjmkfd https://chrome.google.com/webstore/detail/autopatchwork/aeol...

francov88 14y ago

Really cool article - would be amazing to walk through that facility.... love the Google coloured pipes from the pictures

Fando 14y ago

An incredible article!

bhauer 14y ago

All caveats about chrome-dev aside, I find it amusing that this site's navigation does not work in Chrome v24.0.1297.0. Had to use Aurora to view it. Maybe Google really is Sun v2 ("We are the dot in dot-com" == "Where the Internet lives").

Loic 14y ago

I start to be annoyed with the "a power efficiency of 2 is the standard in datacenters". My servers are hosted in a datacenter with a global efficiency of 1.15, proved after more than a year in operation. Announcing that Google is doing 1.2 is simply announcing something wrong and I suppose Google is very happy with this number being provided to the press. It means that some competitor will use it as "Google is the best, they do 1.2, we are at 1.3 we are not too bad", where I bet Google is now near 1.1 or less (they operate without cooling in Belgium for example).

wmf 14y ago
PUE 2 probably was the standard when Google started building their DCs. There's a wide range between enterprisey DCs that consider going from 2 to 1.8 an epic win and clouds/hosters who are getting below 1.3. Google's PUE data since 2008 is published at http://www.google.com/about/datacenters/efficiency/internal/index.html http://www.google.com/about/datacenters/efficiency/internal/...
rryan 14y ago
http://www.google.com/about/datacenters/efficiency/internal/index.html http://www.google.com/about/datacenters/efficiency/internal/... You're right -- those PUE numbers from the article were talking about their PUE at the time. Google's 2012 average PUE across all facilities was 1.12/1.13 with a minimum PUE of 1.09/1.10. Also, Google puts enormous care into the process of calculating PUE since it's kind of black art and if you aren't careful you'll leave out some aspect of your operation that will mislead you into thinking your PUE is lower than it is.

brown9-2 14y ago

In regards to the disaster testing: How did Google do this time? Pretty well. Despite the outages in the corporate network, executive chair Eric Schmidt was able to run a scheduled global all-hands meeting. The imaginary demonstrators were placated by imaginary pizza. How does one decide what will placate imaginary demonstrators? Who calls them off?

ubercore 14y ago
For the purposes of tests like that, they probably just wanted to see that "reasonable action was taken", which will (hand-waving) probably take care of most instances of that type. In the event of real demonstrators, it would just be the opening salvo of damage control, but it's too hard to predicate how a crowd of angry people would react past the first move.

rpearl 14y ago

There are some photos, such as https://www.google.com/about/datacenters/gallery/images/_3000/IDI_018.jpg https://www.google.com/about/datacenters/gallery/images/_300... I wonder why they've mirrored the image (the left side is quite clearly the right side flipped--take a look at the machine identifier labels). What's being hidden?

cbr 14y ago
Perfect symmetry makes it more impressive?
reledi 14y ago
The blue LEDs in the picture you linked to, indicate that the servers are running smoothly. [1] It's possible that some servers were faulty at the time the picture was taken. More likely, it's to make the image look perfect. [1] http://www.google.com/about/datacenters/gallery/#/all/12 http://www.google.com/about/datacenters/gallery/#/all/12

javajosh 14y ago

Hi Google Platform people. Very nice work. As you may know, Randall Monroe (of xkcd fame) has recently started a feature called "what if" on their site. I would like to post a question to you along those lines: What if Google was tasked with building an orbiting datacenter? How about a Dyson ring, or sphere? How would you do it? If we were to use all matter in the solar system for commodity linux hardware, how much gmail storage would I get? How many flops? And what sorts of computation could you do on this monster? Please answer! This should be fun...

alxv 14y ago
Space would be a terrible environment for building a datacenter. The main goals of a datacenter is to make computation as cheap, as fast and as reliable as possible. Having the datacenter orbit around Earth would not help us accomplish any of these goals. First off, building a datacenter in space would not be cheap. It costs around $25,000 to send a kilogram of equipment into a geostationary orbit. [1] So let assume we were to use Dell PowerEdge C1100. Each server costs $14,000 and weights 18kg. [2] This means for each server sent in orbit, you could buy 32 extra ones on Earth. Then, there is the issue of cooling. Although outer space is really cold, its vacuum prevents the heat generated by the machines from being dissipated quickly. Controlling the temperature of a such datacenter would be a very interesting engineering challenge. And then how would you power this datacenter? Converting the excess heat back into electricity could be an interesting option. But most likely, it would need a lot of solar panels. This would make the datacenter cheap to run once built, but the upfront costs would be enormous. And we haven't talked about speed and reliability yet. Since the signal would need to travel about 35,000 km from the geostationary orbit to reach us, communications between Earth and the datacenter would have significant delays. Even at the speed of light, the minimum round trip time would be about 250 milliseconds if we ignore all other possible sources of delay. The hostile space weather would also make it pretty hard to run servers reliability. Radiations would destroy electronics, caused bit to flip randomly and do all bunch of fun stuff to the equipment. But... anyhow! Let's assume anyway that by some magical work of science and Google engineering, we figure ways to manufacture a datacenter directly space for almost nothing by mining the Moon, discover some amazing thermoelectric generators with near 100% efficiency and space shields that blocks almost all radiations. So back our previous example, a high performance PowerEdge gives us up to about 300 GFLOPS of computing power, 192 GB of RAM and 12 TB of storage. Now if we were to convert the total mass of the Moon (7.34767309 × 10²² kg) into one monstrous datacenter, this would give us about 4.0 × 10²¹ servers. It would gives us a whooping 1.2 billions YottaFLOPS (or put differently, 1.2 × 10³³ FLOPS) of compute madness, 0.8 billions YottaBytes and 49 billions YottaBytes of storage. This monster would consume about the equivalent of 1% of the Sun's total power output. [1]: http://www.futron.com/upload/wysiwyg/Resources/Whitepapers/Space_Transportation_Costs_Trends_0902.pdf http://www.futron.com/upload/wysiwyg/Resources/Whitepapers/S... [2]: http://www.dell.com/us/enterprise/p/poweredge-c1100/pd#TechSpec http://www.dell.com/us/enterprise/p/poweredge-c1100/pd#TechS...

dredmorbius 14y ago

When they say that supercomputing is essentially a plumbing problem ... looking at these photos, no kidding.

stock_toaster 14y ago

It is unfortunate (for the rest of us) that datacenter tech is such a competitive advantage for Google. If they were able to share their breakthroughs more readily with others, imagine how much less of the "1.5% of all power globally" datacenters could be using.

no_script 14y ago

Seriously. I can't believe they require JavaScript to view this all this eye-candy and server porn. http://www.google.com/about/datacenters/gallery/ http://www.google.com/about/datacenters/gallery/ I thought GWT was designed to "compile" rendered pages for a wide variety of browsers and permutations of configurations? The pictures are very pretty, but that's really awful of them to release a PR site like this, and force users into using JavaScript. Unforgivable.