5 ms·

"The superhuman folks at the data center are hauling 55 gallon drums of diesel fuel up 17 flights of stairs." I can't for the life of me think why someone woul

by lutze 14y ago

"The superhuman folks at the data center are hauling 55 gallon drums of diesel fuel up 17 flights of stairs." I can't for the life of me think why someone would put a backup generator 17 floors up, these guys don't know you can buy mains cables longer than 4 feet? If it absolutely has to be 17 floors up, buy a fucking pump. Geeks. Sometimes we're so dumb it hurts.

jimmybrite 14y ago
In most buildings, the generator is at the top floor, just saying. Source: I worked @ cbc on Queen street in ottawa,ON, and the generator is on the top floor.
- lutze 14y ago
  Makes sense actually, probably in case of flooding. Doesn't make that pump any less sensible though!
  - mds_ 14y ago
    Apparently the pumps got flooded: http://status.fogcreek.com/2012/10/fog-creek-services-updates.html http://status.fogcreek.com/2012/10/fog-creek-services-update... (near the bottom)
    - lutze 14y ago
      Well, now I do feel like a cunt. Past my bedtime, so didn't connect this with that wee bit of weather the East coast USA experienced recently. Sincerely hope lugging diesel remains the least of their worries.
    - jessaustin 14y ago
      This is reminiscent of what happened at Fukushima Daiichi, although with less dire consequences.
- spolsky 14y ago
  I suspect that's for ventilation reasons
mertd 14y ago
Mind you they are in lower Manhattan, not some random suburban office park.
- rhizome 14y ago
  That was probably their first mistake.
  - mememememememe 14y ago
    LOL I agree. There are a few small data center in Manhattan. Actually, if they choose data center like Amazon they may expect outrage once in a while....
  - tghw 14y ago
    How so? This storm was a whopper. To the point that buoys were reporting waves 5 times taller than anything on record. Historically, Manhattan has been a fine place for a data center. Moreover, Peer1 survived the 2003 blackout, the only other recent event of this magnitude, without service disruption. You can armchair quarterback this all you want, but unless you're actually making the decisions, perhaps you should consider the information available.
    - rhizome 14y ago
      Well sure, and everything is fine until it isn't. It would be impossible for Manhattan to have any kind of Internet without peering and NAPs, but for hosting? I'll choose Nowheresville anytime, which I have done before.
  - dbecker 14y ago
    I'm sure there are situations where you need to get someone to the DC quickly, so a Manhattan DC makes a lot of sense for an NYC based company. The current solution already worked the VAST majority of the time too. If they'd hosted it in central nowhere, people would criticize them for not planning appropriately for the times they need someone at the DC.
nitrogen 14y ago
If you'd read the previous articles about the hurricane's effects on datacenters, you'd know that the diesel pumps in many buildings were wiped out by flooding.
jessaustin 14y ago
"buy a fucking pump" I suspect this is good advice, but the image that popped into my head was an elevated generator, a ground-level tank, and an electric pump reliant on either mains or generator power. "It's always worked during our monthly tests!" Whoops! Probably the elevated generator should be installed with at least a small gravity-fed tank, easily refilled once the pump is operating. I doubt anyone wants a giant fuel depot anywhere inside or on the roof of the building. Keep in mind that for heights taller than 3 floors (and practically, the limit is shorter than that), pumps have to be located on the same level as the fuel supply because the atmospheric column only has so much mass. Pumps and their electrical inputs should probably be installed with waterproofing if flooding is considered likely.
- X-Istence 14y ago
  The fuel is stored in the basement of the datacenter. That flooded, taking the pump that was supposed to pump up the fuel stored in the basement to the 17th floor. This has nothing to do with buying a pump, that was already there, this has to do with the fact that the diesel tanks and the pumps for them are under water. In the future I hope they put the pumps on higher floors, and make sure the fuel tanks are completely sealed, so even if the tanks are flooded the pump can keep pumping fuel out of it.
rexreed 14y ago
From http://status.fogcreek.com/2012/10/fog-creek-services-updates.html http://status.fogcreek.com/2012/10/fog-creek-services-update...: "Here's the physical situation: The generators are on a high floor in the building and the pumps supplying the generators with fuel are submerged. The best option at this point is for people to physically lug diesel up over a dozen floors, or make other arrangements for pumping fuel to that high floor. " Yes, a generator not in the basement / ground level makes sense. When a flood happens, it usually happens from the ground floor up ;) Thus why generators are at the upper levels. Even in this case, using a pump solution makes little sense. Assuming you have electricity to power the pump, I suppose you could power the pump with the electricity from the remaining fuel, but a 17 floor pipe containing fuel would probably be heavier (and more dangerous) than a long mains power cord, and it wouldn't make sense to have a 17 floor cable to power the pump with the pump at the bottom of the run than it would to have just the pipe of fuel running the length of 17 floors and the pump at the top of the run. Time for a winch? I don't think Home Depot stocks a 17 story pipe, and relocating the generator probably will take more time than just winching / carrying up the barrels.
- nitrogen 14y ago
  I believe having the pumps at the bottom makes sense because you only have ~14PSI of atmospheric pressure to use to pull the fuel up if the pump were at the top, but you can generate as much pressure as you need at the bottom to push the fuel up.
- jlgreco 14y ago
  Generators higher up makes sense, but 17 stories high? I suppose real-estate concerns limit where you can put them though.
  - mhp 14y ago
    In this case, I think you are correct. Need lots of exhaust so have to be on the roof. One police plaza just tore the side of their building off to make room for generator exhaust. It's a warzone in lower manhattan right now.

by lutze 14y ago

jimmybrite 14y ago

In most buildings, the generator is at the top floor, just saying. Source: I worked @ cbc on Queen street in ottawa,ON, and the generator is on the top floor.

lutze 14y ago
Makes sense actually, probably in case of flooding. Doesn't make that pump any less sensible though!
- mds_ 14y ago
  Apparently the pumps got flooded: http://status.fogcreek.com/2012/10/fog-creek-services-updates.html http://status.fogcreek.com/2012/10/fog-creek-services-update... (near the bottom)
  - lutze 14y ago
    Well, now I do feel like a cunt. Past my bedtime, so didn't connect this with that wee bit of weather the East coast USA experienced recently. Sincerely hope lugging diesel remains the least of their worries.
  - jessaustin 14y ago
    This is reminiscent of what happened at Fukushima Daiichi, although with less dire consequences.
spolsky 14y ago
I suspect that's for ventilation reasons

mertd 14y ago

Mind you they are in lower Manhattan, not some random suburban office park.

rhizome 14y ago
That was probably their first mistake.
- mememememememe 14y ago
  LOL I agree. There are a few small data center in Manhattan. Actually, if they choose data center like Amazon they may expect outrage once in a while....
- tghw 14y ago
  How so? This storm was a whopper. To the point that buoys were reporting waves 5 times taller than anything on record. Historically, Manhattan has been a fine place for a data center. Moreover, Peer1 survived the 2003 blackout, the only other recent event of this magnitude, without service disruption. You can armchair quarterback this all you want, but unless you're actually making the decisions, perhaps you should consider the information available.
  - rhizome 14y ago
    Well sure, and everything is fine until it isn't. It would be impossible for Manhattan to have any kind of Internet without peering and NAPs, but for hosting? I'll choose Nowheresville anytime, which I have done before.
- dbecker 14y ago
  I'm sure there are situations where you need to get someone to the DC quickly, so a Manhattan DC makes a lot of sense for an NYC based company. The current solution already worked the VAST majority of the time too. If they'd hosted it in central nowhere, people would criticize them for not planning appropriately for the times they need someone at the DC.

nitrogen 14y ago

If you'd read the previous articles about the hurricane's effects on datacenters, you'd know that the diesel pumps in many buildings were wiped out by flooding.

jessaustin 14y ago

"buy a fucking pump" I suspect this is good advice, but the image that popped into my head was an elevated generator, a ground-level tank, and an electric pump reliant on either mains or generator power. "It's always worked during our monthly tests!" Whoops! Probably the elevated generator should be installed with at least a small gravity-fed tank, easily refilled once the pump is operating. I doubt anyone wants a giant fuel depot anywhere inside or on the roof of the building. Keep in mind that for heights taller than 3 floors (and practically, the limit is shorter than that), pumps have to be located on the same level as the fuel supply because the atmospheric column only has so much mass. Pumps and their electrical inputs should probably be installed with waterproofing if flooding is considered likely.

X-Istence 14y ago
The fuel is stored in the basement of the datacenter. That flooded, taking the pump that was supposed to pump up the fuel stored in the basement to the 17th floor. This has nothing to do with buying a pump, that was already there, this has to do with the fact that the diesel tanks and the pumps for them are under water. In the future I hope they put the pumps on higher floors, and make sure the fuel tanks are completely sealed, so even if the tanks are flooded the pump can keep pumping fuel out of it.

rexreed 14y ago

From http://status.fogcreek.com/2012/10/fog-creek-services-updates.html http://status.fogcreek.com/2012/10/fog-creek-services-update...: "Here's the physical situation: The generators are on a high floor in the building and the pumps supplying the generators with fuel are submerged. The best option at this point is for people to physically lug diesel up over a dozen floors, or make other arrangements for pumping fuel to that high floor. " Yes, a generator not in the basement / ground level makes sense. When a flood happens, it usually happens from the ground floor up ;) Thus why generators are at the upper levels. Even in this case, using a pump solution makes little sense. Assuming you have electricity to power the pump, I suppose you could power the pump with the electricity from the remaining fuel, but a 17 floor pipe containing fuel would probably be heavier (and more dangerous) than a long mains power cord, and it wouldn't make sense to have a 17 floor cable to power the pump with the pump at the bottom of the run than it would to have just the pipe of fuel running the length of 17 floors and the pump at the top of the run. Time for a winch? I don't think Home Depot stocks a 17 story pipe, and relocating the generator probably will take more time than just winching / carrying up the barrels.

nitrogen 14y ago
I believe having the pumps at the bottom makes sense because you only have ~14PSI of atmospheric pressure to use to pull the fuel up if the pump were at the top, but you can generate as much pressure as you need at the bottom to push the fuel up.
jlgreco 14y ago
Generators higher up makes sense, but 17 stories high? I suppose real-estate concerns limit where you can put them though.
- mhp 14y ago
  In this case, I think you are correct. Need lots of exhaust so have to be on the roof. One police plaza just tore the side of their building off to make room for generator exhaust. It's a warzone in lower manhattan right now.