5 ms·

Yes but if you have reliable power from two different sources then the biggest risk (I'd imagine) is the failover circuitry! Something that should be tested tbh

by ItsBob 5y ago

Yes but if you have reliable power from two different sources then the biggest risk (I'd imagine) is the failover circuitry! Something that should be tested tbh. Also, there are banks of batteries and generators in between the power company cables and the kit: did they not kick-in? Again, this is all pure speculation: I have absolutely no idea of the exact failure, nor how their infrastructure is held together - this is all just speculation for the hell of it :)

merlyn 5y ago
Frying hardware can affect much wider scope. I've had bad power supplies fry out taking the whole power circuit with it, and thus half (or whatever fraction) of the rack's power. I've also had bad power supplies bring down the whole machine as they shunted everything internal too. When things go bad, anything can happen. You can provide the best effort, and it'll usually work as expected, but there will always be something that can overcome your best efforts.
notyourday 5y ago
> Yes but if you have reliable power from two different sources then the biggest risk (I'd imagine) is the failover circuitry! Something that should be tested tbh. That's ATS. It is not really advisable to test their under load performance because the failure of an ATS would be catastrophic. ATS typically would be tested at the installation and after that their parameters would be monitored. Replacing a functional in line ATS would be a 9-12 months long project. > Also, there are banks of batteries and generators in between the power company cables and the kit: did they not kick-in? At high energy you are pretty much always going to use an ATS.
- belfalas 5y ago
  > the failure of an ATS would be catastrophic Because that would mean no power at all to the DC and no way to get it back? (I am completely ignorant on this topic)
  - notyourday 5y ago
    > Because that would mean no power at all to the DC and no way to get it back? (I am completely ignorant on this topic) While most of smarts in the ATS are in the electronics, the really nasty failures come from the mechanical part. At the end of the day a high energy ATS looks just like a switch behind a meter in your house. There's a lip that goes from one position to another, except in a high energy ATS the lip is big and when the transfer occurs it slams from one source to another. There are only so many of those physical slams that it can withstand to being with so you want to minimize that number. The second failure mode is that after transfer to non-main source, the lip can get stuck there, making it impossible to switch back on the main. [Once I have seem the lip melt into the secondary position. While I thought it was weird, the guys from the power company said it is not that uncommon.] This creates a massive problem as the non-main source is typically not designed for long term 24x7 operation. So now you are stuck on a secondary feeding system and you cant just transfer to main without de-energizing the system i.e. taking the power out of the entire data center.

by ItsBob 5y ago

merlyn 5y ago

Frying hardware can affect much wider scope. I've had bad power supplies fry out taking the whole power circuit with it, and thus half (or whatever fraction) of the rack's power. I've also had bad power supplies bring down the whole machine as they shunted everything internal too. When things go bad, anything can happen. You can provide the best effort, and it'll usually work as expected, but there will always be something that can overcome your best efforts.

notyourday 5y ago

> Yes but if you have reliable power from two different sources then the biggest risk (I'd imagine) is the failover circuitry! Something that should be tested tbh. That's ATS. It is not really advisable to test their under load performance because the failure of an ATS would be catastrophic. ATS typically would be tested at the installation and after that their parameters would be monitored. Replacing a functional in line ATS would be a 9-12 months long project. > Also, there are banks of batteries and generators in between the power company cables and the kit: did they not kick-in? At high energy you are pretty much always going to use an ATS.

belfalas 5y ago
> the failure of an ATS would be catastrophic Because that would mean no power at all to the DC and no way to get it back? (I am completely ignorant on this topic)
- notyourday 5y ago
  > Because that would mean no power at all to the DC and no way to get it back? (I am completely ignorant on this topic) While most of smarts in the ATS are in the electronics, the really nasty failures come from the mechanical part. At the end of the day a high energy ATS looks just like a switch behind a meter in your house. There's a lip that goes from one position to another, except in a high energy ATS the lip is big and when the transfer occurs it slams from one source to another. There are only so many of those physical slams that it can withstand to being with so you want to minimize that number. The second failure mode is that after transfer to non-main source, the lip can get stuck there, making it impossible to switch back on the main. [Once I have seem the lip melt into the secondary position. While I thought it was weird, the guys from the power company said it is not that uncommon.] This creates a massive problem as the non-main source is typically not designed for long term 24x7 operation. So now you are stuck on a secondary feeding system and you cant just transfer to main without de-energizing the system i.e. taking the power out of the entire data center.