3 ms·

Unfortunately, physics dictates this setup. To get to a top floor generator (which is generally necessary for ventilation), you have to pump from the bottom. Fo

by tghw 14y ago

Unfortunately, physics dictates this setup. To get to a top floor generator (which is generally necessary for ventilation), you have to pump from the bottom. For most buildings, that means from the basement, where the fuel tanks are placed for other obvious reasons. (Consider if one started leaking, would you want it dripping down to all of the floors below it?) These data centers are well designed, but it's impossible to cover every disaster scenario. It's easy for you to sit here after the fact and snipe at them. If you really think you can do better, go start designing them yourself. If you succeed, you'll do quite well for yourself.

danudey 14y ago
Regardless of whether they can be designed better, the data centres were assuring everyone that everything will be fine because they have backup generators, ignoring the probability that flooding would take out those generators. If I had a website or service hosted at a data centre likely to go offline, I'd be annoyed if I were reassured that everything was taken care of right before the system went offline.
DannyBee 14y ago
Your one complaint about diesel fuel dripping is not a big deal in the short term. Diesel fuel is quite safe and hard to ignite. THere are plenty of places/situations where significantly more toxic substances are stored in tanks that are not in basements. Your logic would apply more there (could you imagine if acetone was dripping from the ceiling?). As a practical matter, anything leaking through floors is really bad, but the problem is solvable and not a good reason to keep fuel tanks in a basement. So what other reasons are fuel tanks placed in basements that outweigh the risk of putting them in the basement (everything except tornadoes would seem to be a con for basements)? Additionally, "Flooding" would seem to be a fairly typical disaster scenario. Again, what does putting them in the basement prevent against that makes it a good alternative. Plenty of people are designing better datacenters and using them, there is no need for the parent to go out and do anything. If customers hold internap/et al accountable, the problem should take care of itself. (of course, they won't, but that's another matter :P)
- HeyLaughingBoy 14y ago
  It's easy to say diesel fuel is "quite safe and hard to ignite." But if the NYC Fire Code says you have to store it in the basement, then guess what? You have to store it in the basement
  - DannyBee 14y ago
    Then either 1. Stop claiming they are disaster tolerant, because they aren't. Flooding is a common case. The exact rule i'm aware of is "406.5.3 Storage tanks. Motor fuel storage tanks shall be installed below ground, except as authorized by the rules of the Fire Department" So get the fire department to authorize it. 2. Datacener folks should lobby new york city. Bloomberg would likely care. 3. Find something other than motor fuel, which is what is restricted :) If all else fails: 4. Stop building disaster tolerant datacenters in new york. This is why a lot of financial folks who aren't HFT's build in Seacaucus, etc. There are very few people who need to place a DC in new york, and can't afford to be across the river. (I'm very keenly aware of the bandwidth situation in both places, FWIW) I'm sure if datacenters start moving across the river, Bloomberg would care. Building codes and other restrictions are taken into account by the big boys (Google, Apple, Facebook, etc) when they build large datacenters.
jmillikin 14y ago
Fuel pumps can be installed such that they are resistant to flooding. A water-tight seal on the fuel tank, sealed power line from the generator, and other straightforward water-proofing measure would have kept Peer 1's generator working as long as the building's fuel supply held out. Manhattan is a low-elevation coastal island in a region of the country prone to hurricanes. Flooding is an obvious disaster scenario. It's not like we're asking them to plan for nuclear war, here.
grey-area 14y ago
There's no reason the fuel tanks and pumps can't be in the basement and still withstand a flood if they're waterproof, which would be relatively easy to achieve. I suppose these systems are typically designed to mitigate blackouts though rather than natural distasters, and floods were not considered as a possibility. As you say, it's easy to be critical after the fact.
illumin8 14y ago
How about coming to the reasonable conclusion that a data center in a 17 story tall building is not practical? Most purpose built data centers are only a few stories tall and don't have these type of issues as they are not built on islands close to sea level. For those companies hosting production services on an island a few feet from sea level without a DR plan, I simply laugh. You deserve our scorn, not our production data.

by tghw 14y ago

danudey 14y ago

Regardless of whether they can be designed better, the data centres were assuring everyone that everything will be fine because they have backup generators, ignoring the probability that flooding would take out those generators. If I had a website or service hosted at a data centre likely to go offline, I'd be annoyed if I were reassured that everything was taken care of right before the system went offline.

DannyBee 14y ago

Your one complaint about diesel fuel dripping is not a big deal in the short term. Diesel fuel is quite safe and hard to ignite. THere are plenty of places/situations where significantly more toxic substances are stored in tanks that are not in basements. Your logic would apply more there (could you imagine if acetone was dripping from the ceiling?). As a practical matter, anything leaking through floors is really bad, but the problem is solvable and not a good reason to keep fuel tanks in a basement. So what other reasons are fuel tanks placed in basements that outweigh the risk of putting them in the basement (everything except tornadoes would seem to be a con for basements)? Additionally, "Flooding" would seem to be a fairly typical disaster scenario. Again, what does putting them in the basement prevent against that makes it a good alternative. Plenty of people are designing better datacenters and using them, there is no need for the parent to go out and do anything. If customers hold internap/et al accountable, the problem should take care of itself. (of course, they won't, but that's another matter :P)

HeyLaughingBoy 14y ago
It's easy to say diesel fuel is "quite safe and hard to ignite." But if the NYC Fire Code says you have to store it in the basement, then guess what? You have to store it in the basement
- DannyBee 14y ago
  Then either 1. Stop claiming they are disaster tolerant, because they aren't. Flooding is a common case. The exact rule i'm aware of is "406.5.3 Storage tanks. Motor fuel storage tanks shall be installed below ground, except as authorized by the rules of the Fire Department" So get the fire department to authorize it. 2. Datacener folks should lobby new york city. Bloomberg would likely care. 3. Find something other than motor fuel, which is what is restricted :) If all else fails: 4. Stop building disaster tolerant datacenters in new york. This is why a lot of financial folks who aren't HFT's build in Seacaucus, etc. There are very few people who need to place a DC in new york, and can't afford to be across the river. (I'm very keenly aware of the bandwidth situation in both places, FWIW) I'm sure if datacenters start moving across the river, Bloomberg would care. Building codes and other restrictions are taken into account by the big boys (Google, Apple, Facebook, etc) when they build large datacenters.

jmillikin 14y ago

Fuel pumps can be installed such that they are resistant to flooding. A water-tight seal on the fuel tank, sealed power line from the generator, and other straightforward water-proofing measure would have kept Peer 1's generator working as long as the building's fuel supply held out. Manhattan is a low-elevation coastal island in a region of the country prone to hurricanes. Flooding is an obvious disaster scenario. It's not like we're asking them to plan for nuclear war, here.

grey-area 14y ago

There's no reason the fuel tanks and pumps can't be in the basement and still withstand a flood if they're waterproof, which would be relatively easy to achieve. I suppose these systems are typically designed to mitigate blackouts though rather than natural distasters, and floods were not considered as a possibility. As you say, it's easy to be critical after the fact.

illumin8 14y ago

How about coming to the reasonable conclusion that a data center in a 17 story tall building is not practical? Most purpose built data centers are only a few stories tall and don't have these type of issues as they are not built on islands close to sea level. For those companies hosting production services on an island a few feet from sea level without a DR plan, I simply laugh. You deserve our scorn, not our production data.