7 ms·

So basically, my takeaway is: * Freeflowing air in DCs is good for cooling( * It’s bad when you have a fire; * An improved DC design would allow an operator t

So basically, my takeaway is: * Freeflowing air in DCs is good for cooling( * It’s bad when you have a fire; * An improved DC design would allow an operator to $somehow stop the freeflowing air (although one could argue that it’s not free flowing anymore if one can control it); * I’d like to know how much money really was saved by not allowing the UPSes to be cut off. I’m very curious how the insurance companies respond, and whether they’ll demand e.g. UPS supplies to be able to cut off. Or maybe in general, the fire department should be more aware of these types of trade-offs being made, and give their approval accordingly.

lbriner 5y ago
It is VERY common in buildings to have ventilation dampers that are closed when a fire alarm starts exactly for this reason. I am most surprised that the design sounds like it was made a certain way deliberately even though the basics were got wrong. Does Strasbourg not have any fire regulations that require fireman accessible power cut-outs, automatic extinguishers etc? It doesn't matter just that their equipment burned but the severity of the fire was massively dangerous to the fire brigade and anything else nearby if it couldn't be extinguished properly.
- ovi256 5y ago
  > Does Strasbourg not have any fire regulations that require fireman accessible power cut-outs, automatic extinguishers etc? I think it's a safe bet it doesn't, because that building passed fire safety review and was approved as such.
  - kube-system 5y ago
    I think it’s a valid question; inspections themselves can be failure points.
  - michaelt 5y ago
    I can't speak for Strasbourg, but here in the UK you don't get an inspection by the local building inspector or fire department - instead, you hire a fire risk assessor of your choice, on the free market. So if you're not in compliance, you don't need to bribe anyone or cheat - you simply have to hire a friendly dumbass who'll check the basics but won't ask too many questions.
    - xxpor 5y ago
      Mark this one up as something else that's been privatized in the UK that's shocking to me as an American. This sort of stuff is always run through the government (the level depends on the state). Insurance companies might impose additional requirements, but then they'll send out their own inspectors.
      - j16sdiz 5y ago
        The inspector is a chartered engineer, and is liable if your building don't confirm to the fire code.
    - throwaway0a5e 5y ago
      >Instead, you hire a fire risk assessor of your choice, on the free market. That sounds a lot better than the one size fits all crap we get in the US. If you are trying to do anything other than take over an existing and operating industrial site that's grandfathered in or run a warehouse for some sort of boring and non-reactive goods it tends to be a terrible maze of conflicting and irrelevant requirements.
      - xxpor 5y ago
        Note the OP said the inspectors were privatized, not the actual building codes. The only difference is the private inspectors sound like they'll let you skate by on some stuff. You find the same dynamic in states with private emissions inspections. Spend any time on a car enthusiast website and everyone knows "the guy" to go to that'll pass your obviously illegal emissions equipment bypass...
        throwaway0a5e 5y ago
        >Note the OP said the inspectors were privatized I know exactly what he said and that's why I said exactly what I said. >The only difference is the private inspectors sound like they'll let you skate by on some stuff Government building inspections are incredibly onerous unless you're a homeowner building a deck or a developer building cookie cutter houses or offices. You run into all sorts of stupid edge cases depending on what the facility being built is intended for and you need to get clarification on what you should do. Have you ever actually tried to get government clarification about something? They need to be dragged kicking and screaming, often through a courtroom to ever clarify anything in writing because nobody wants to set a precedent or take the responsibility of making judgement calls. Having professionals who will take the responsibility of those judgement calls is a massive net plus. The fact that your knee jerk response is to frame professionals using judgement their jobs as "letting you state by" is just crazy. Would you say the same thing if the context was a PE crunching numbers, finding something that didn't pass the default rules and then using their judgement to conclude it was fine because of other situational details? People do their jobs satisfactorily the overwhelming majority of the time. Being on the government payroll doesn't make public inspectors immune from playing favorites or making screw ups. It just means the oversights follow a different pattern. Everyone's experienced plenty of government inspectors who don't do their job because it's Friday and they want to GTFO. > Spend any time on a car enthusiast website and everyone knows "the guy" to go to that'll pass your obviously illegal emissions equipment bypass... If the vehicle code were written the way the building code and occupational health and safety code is that's the only way any commercial vehicle that isn't a cookie cutter box truck would ever pass an inspection.
- mschuster91 5y ago
  > Does Strasbourg not have any fire regulations that require fireman accessible power cut-outs, automatic extinguishers etc? Fire codes for industrial buildings, office buildings and residential buildings differ by quite a lot. Usually, they get stricter the more people are supposed to be in a building and what their training status in firefighting/escape is.
- Nextgrid 5y ago
  > Does Strasbourg not have any fire regulations that require fireman accessible power cut-outs, automatic extinguishers etc? On the other hand, it could be that this was deliberately allowed for buildings with low/no occupancy, easy, safe & quick escape routes and no risk to neighbouring properties. If someone wants to risk their own property, as long as they don't risk harming anyone, why not allow it?
  - wmf 5y ago
    Just don't try to buy fire insurance.
  - waste_monk 5y ago
    >If someone wants to risk their own property, as long as they don't risk harming anyone, why not allow it? Because it's a flawed premise, you don't exist in isolation from the outside world. The smoke is harmful to everyone nearby, especially a data center fire where you have burning servers, UPS batteries, etc. contributing large quantities of burning cadmium, lead, mercury, assorted plastics and so on. The environmental impact from the fire event alone should make this a non-starter. Also, little to nothing can be recycled after such an event, and replacing the lost equipment requires a whole bunch of environmentally damaging mining and manufacturing processes. >no risk to neighbouring properties. Even if you plan carefully there is an unacceptable risk of fire spreading. And while some datacenters are moving to the countryside or aquatic environments, most are in built up areas for easy access to industrial power supply, workers, spare parts, freight, etc... And I have no confidence that a company who's disaster plan is "screw it, let it all burn" would adequately maintain firebreaks and other containment measures. Frankly, I find the very idea of such excessive and deliberately considered wastefulness to be grotesque.
smarx007 5y ago
> I’d like to know how much money really was saved by not allowing the UPSes to be cut off. I don't think this was a cost-cutting measure. Instead, (automatic) cut-off devices are also systems that can fail. And if there is such a centralized device, it can nullify the hard work of 4 redundant power supplies if it "decides" to malfunction one day. So, this was a SPOF-cutting measure. I still think that having 4 separate switches for 4 power supplies is not so wrong. Firefighters anyway wrote that the electrical room had a door at 400C and observed 1-meter arcs, so they had to ask the electrical company to cut the power from their side for the sake of safety anyway.
- tssva 5y ago
  I disagree with your idea that not having an automated power disconnect system is acceptable but for the sake of argument lets say it is. The standard for manual emergency power cutoff switches is to be located where they are easily accessible in an emergency which means outside of the electrical room. Most large data centers have manual switches in multiple locations. They should at a minimum have had cutoff switches located in their NOC.
  - smarx007 5y ago
    Right! Sorry, I didn't write it clearly but I agree that there should be automatic disconnect, but think it's OK to have 4 separate disconnects to maintain the supply redundancy. Though now it can get really expensive.
- the_arun 5y ago
  Building datacenters and is hard. So this is an opportunity for cloud providers. 1. Imagine if Amazon starts building datacenters for others as a service 2. People starts leasing or renting entire DCs built by Amazon, Google etc. Maintenance is done by these cloud providers. Do we have these already?
  - traceroute66 5y ago
    > 1. Imagine if Amazon starts building datacenters for others as a service Just as an FYI, those fancy datacentres you see on Amazon/Google/Microsoft marketing videos are really the exception rather than the rule. You will find the majority of cloud servers in exactly the same third-party datacentres that everyone else uses. Why ? Because only the US and perhaps one or two other countries in the world has the spare land for a cloud operator to dump a massive datacentre campus on. Most other countries don't. Or if they do, its either protected land (greenbelt etc.) or its uninhabitable (e.g. Australian Outback).

lbriner 5y ago

It is VERY common in buildings to have ventilation dampers that are closed when a fire alarm starts exactly for this reason. I am most surprised that the design sounds like it was made a certain way deliberately even though the basics were got wrong. Does Strasbourg not have any fire regulations that require fireman accessible power cut-outs, automatic extinguishers etc? It doesn't matter just that their equipment burned but the severity of the fire was massively dangerous to the fire brigade and anything else nearby if it couldn't be extinguished properly.

ovi256 5y ago
> Does Strasbourg not have any fire regulations that require fireman accessible power cut-outs, automatic extinguishers etc? I think it's a safe bet it doesn't, because that building passed fire safety review and was approved as such.
- kube-system 5y ago
  I think it’s a valid question; inspections themselves can be failure points.
- michaelt 5y ago
  I can't speak for Strasbourg, but here in the UK you don't get an inspection by the local building inspector or fire department - instead, you hire a fire risk assessor of your choice, on the free market. So if you're not in compliance, you don't need to bribe anyone or cheat - you simply have to hire a friendly dumbass who'll check the basics but won't ask too many questions.
  - xxpor 5y ago
    Mark this one up as something else that's been privatized in the UK that's shocking to me as an American. This sort of stuff is always run through the government (the level depends on the state). Insurance companies might impose additional requirements, but then they'll send out their own inspectors.
    - j16sdiz 5y ago
      The inspector is a chartered engineer, and is liable if your building don't confirm to the fire code.
  - throwaway0a5e 5y ago
    >Instead, you hire a fire risk assessor of your choice, on the free market. That sounds a lot better than the one size fits all crap we get in the US. If you are trying to do anything other than take over an existing and operating industrial site that's grandfathered in or run a warehouse for some sort of boring and non-reactive goods it tends to be a terrible maze of conflicting and irrelevant requirements.
    - xxpor 5y ago
      Note the OP said the inspectors were privatized, not the actual building codes. The only difference is the private inspectors sound like they'll let you skate by on some stuff. You find the same dynamic in states with private emissions inspections. Spend any time on a car enthusiast website and everyone knows "the guy" to go to that'll pass your obviously illegal emissions equipment bypass...
      - throwaway0a5e 5y ago
        >Note the OP said the inspectors were privatized I know exactly what he said and that's why I said exactly what I said. >The only difference is the private inspectors sound like they'll let you skate by on some stuff Government building inspections are incredibly onerous unless you're a homeowner building a deck or a developer building cookie cutter houses or offices. You run into all sorts of stupid edge cases depending on what the facility being built is intended for and you need to get clarification on what you should do. Have you ever actually tried to get government clarification about something? They need to be dragged kicking and screaming, often through a courtroom to ever clarify anything in writing because nobody wants to set a precedent or take the responsibility of making judgement calls. Having professionals who will take the responsibility of those judgement calls is a massive net plus. The fact that your knee jerk response is to frame professionals using judgement their jobs as "letting you state by" is just crazy. Would you say the same thing if the context was a PE crunching numbers, finding something that didn't pass the default rules and then using their judgement to conclude it was fine because of other situational details? People do their jobs satisfactorily the overwhelming majority of the time. Being on the government payroll doesn't make public inspectors immune from playing favorites or making screw ups. It just means the oversights follow a different pattern. Everyone's experienced plenty of government inspectors who don't do their job because it's Friday and they want to GTFO. > Spend any time on a car enthusiast website and everyone knows "the guy" to go to that'll pass your obviously illegal emissions equipment bypass... If the vehicle code were written the way the building code and occupational health and safety code is that's the only way any commercial vehicle that isn't a cookie cutter box truck would ever pass an inspection.
mschuster91 5y ago
> Does Strasbourg not have any fire regulations that require fireman accessible power cut-outs, automatic extinguishers etc? Fire codes for industrial buildings, office buildings and residential buildings differ by quite a lot. Usually, they get stricter the more people are supposed to be in a building and what their training status in firefighting/escape is.
Nextgrid 5y ago
> Does Strasbourg not have any fire regulations that require fireman accessible power cut-outs, automatic extinguishers etc? On the other hand, it could be that this was deliberately allowed for buildings with low/no occupancy, easy, safe & quick escape routes and no risk to neighbouring properties. If someone wants to risk their own property, as long as they don't risk harming anyone, why not allow it?
- wmf 5y ago
  Just don't try to buy fire insurance.
- waste_monk 5y ago
  >If someone wants to risk their own property, as long as they don't risk harming anyone, why not allow it? Because it's a flawed premise, you don't exist in isolation from the outside world. The smoke is harmful to everyone nearby, especially a data center fire where you have burning servers, UPS batteries, etc. contributing large quantities of burning cadmium, lead, mercury, assorted plastics and so on. The environmental impact from the fire event alone should make this a non-starter. Also, little to nothing can be recycled after such an event, and replacing the lost equipment requires a whole bunch of environmentally damaging mining and manufacturing processes. >no risk to neighbouring properties. Even if you plan carefully there is an unacceptable risk of fire spreading. And while some datacenters are moving to the countryside or aquatic environments, most are in built up areas for easy access to industrial power supply, workers, spare parts, freight, etc... And I have no confidence that a company who's disaster plan is "screw it, let it all burn" would adequately maintain firebreaks and other containment measures. Frankly, I find the very idea of such excessive and deliberately considered wastefulness to be grotesque.

smarx007 5y ago

> I’d like to know how much money really was saved by not allowing the UPSes to be cut off. I don't think this was a cost-cutting measure. Instead, (automatic) cut-off devices are also systems that can fail. And if there is such a centralized device, it can nullify the hard work of 4 redundant power supplies if it "decides" to malfunction one day. So, this was a SPOF-cutting measure. I still think that having 4 separate switches for 4 power supplies is not so wrong. Firefighters anyway wrote that the electrical room had a door at 400C and observed 1-meter arcs, so they had to ask the electrical company to cut the power from their side for the sake of safety anyway.

tssva 5y ago
I disagree with your idea that not having an automated power disconnect system is acceptable but for the sake of argument lets say it is. The standard for manual emergency power cutoff switches is to be located where they are easily accessible in an emergency which means outside of the electrical room. Most large data centers have manual switches in multiple locations. They should at a minimum have had cutoff switches located in their NOC.
- smarx007 5y ago
  Right! Sorry, I didn't write it clearly but I agree that there should be automatic disconnect, but think it's OK to have 4 separate disconnects to maintain the supply redundancy. Though now it can get really expensive.
the_arun 5y ago
Building datacenters and is hard. So this is an opportunity for cloud providers. 1. Imagine if Amazon starts building datacenters for others as a service 2. People starts leasing or renting entire DCs built by Amazon, Google etc. Maintenance is done by these cloud providers. Do we have these already?
- traceroute66 5y ago
  > 1. Imagine if Amazon starts building datacenters for others as a service Just as an FYI, those fancy datacentres you see on Amazon/Google/Microsoft marketing videos are really the exception rather than the rule. You will find the majority of cloud servers in exactly the same third-party datacentres that everyone else uses. Why ? Because only the US and perhaps one or two other countries in the world has the spare land for a cloud operator to dump a massive datacentre campus on. Most other countries don't. Or if they do, its either protected land (greenbelt etc.) or its uninhabitable (e.g. Australian Outback).