12 ms·

https://twitter.com/olesovhcom/status/1369504527544705025 https://twitter.com/olesovhcom/status/1369504527544705025 "Update 5:20pm. Everybody is safe. Fire de

https://twitter.com/olesovhcom/status/1369504527544705025 https://twitter.com/olesovhcom/status/1369504527544705025 "Update 5:20pm. Everybody is safe. Fire destroyed SBG2. A part of SBG1 is destroyed. Firefighters are protecting SBG3. no impact SBG4."

jen20 6y ago
An easy mistake (in the quoted tweet) to make given the circumstances, but it probably should be 5:20am rather than pm.
akamaka 6y ago
For some context: “SBG1, the first Strasbourg data center, consisting of twelve containers, came online in 2012. The 12 containers had a capacity of 12,000 servers. SBG2 is a non-container data center in 2016 using its “Tower” design with a capacity of 30,000 servers. SBG3 tower was built in 2017 with a capacity of 30,000 servers. SBG4 was built in 2013 as several containers to augment capacity, but was decommissioned in 2018 and moved to SB3” https://baxtel.com/data-center/ovh-strasbourg-campus https://baxtel.com/data-center/ovh-strasbourg-campus
- Scoundreller 6y ago
  and a 2017 (!) article where they were planning to remove 1 and 2 because of power issues! > OVH to Disassemble Container Data Centers after Epic Outage in Europe > “This is probably the worst-case scenario that could have happened to us.” > OVH [...] is planning to shut down and disassemble two of the three data centers on its campus in Strasbourg, France, following a power outage that brought down the entire campus Friday, causing prolonged disruption to customer applications that lasted throughout the day and well into the evening. https://www.datacenterknowledge.com/uptime/ovh-disassemble-container-data-centers-after-epic-outage-europe https://www.datacenterknowledge.com/uptime/ovh-disassemble-c...
  - Macha 6y ago
    It sounds like the plan was to shutdown 1 and 4, the latter of which happened and the former which did not.
    - Scoundreller 6y ago
      It’s confusing as they say they’re shutting down 2 of the 3, but then there’s 4...
AdamJacobMuller 6y ago
> Fire destroyed SBG2 This is crazy. SBG2 was HUGE and if this isn't a translation error on the part of Octave (which I could understand given the stress and ESL) I have a hard time fathoming what kind of fire could destroy a whole facility with nearly 1000 racks of equipment spread out across separated halls. I'm really hoping "destroyed" here means "we lost all power and network core and there's smoke/fire/physical damage to SOME of that" I can't even fathom a worst-case scenario of a transformer explosion (which does occur and I've seen the aftermath of) having this big of an impact. Datacenters are built to contain and mitigate these kinds of issues. Fire breaks, dry-pipe sprinkler systems and fire-extinguishing gas systems are all designed to prevent a fire from becoming large-scale. Really glad nobody was hurt. OVH is gonna have a bad time cleaning all this up.
- qbasic_forever 6y ago
  Just guessing, but maybe a fire suppression system going off could wipe out all the machines? The couple datacenters I've been inside were small, old and used halon gas which wasn't supposed to destroy the machines. No idea how it works in big places these days.
  - atkbrah 6y ago
    A few years back there was an incident in Sweden where noise coming from the gas based fire suppression system going off destroyed hard drives [1]. 1. https://www.theregister.com/2018/04/26/decibels_destroy_disk_drives_in_digiplex_data_centre/ https://www.theregister.com/2018/04/26/decibels_destroy_disk...
    - abrookewood 6y ago
      I've also seen a weird video (lost to time unfortunately), where someone showed that they could yell at their servers in a data centre and introduce errors (or something similar). Was very strange to see, but they had a console up clearly showing that it was having an impact.
      - 6y ago
deleted 6y ago
[deleted]

jen20 6y ago

An easy mistake (in the quoted tweet) to make given the circumstances, but it probably should be 5:20am rather than pm.

akamaka 6y ago

For some context: “SBG1, the first Strasbourg data center, consisting of twelve containers, came online in 2012. The 12 containers had a capacity of 12,000 servers. SBG2 is a non-container data center in 2016 using its “Tower” design with a capacity of 30,000 servers. SBG3 tower was built in 2017 with a capacity of 30,000 servers. SBG4 was built in 2013 as several containers to augment capacity, but was decommissioned in 2018 and moved to SB3” https://baxtel.com/data-center/ovh-strasbourg-campus https://baxtel.com/data-center/ovh-strasbourg-campus

Scoundreller 6y ago
and a 2017 (!) article where they were planning to remove 1 and 2 because of power issues! > OVH to Disassemble Container Data Centers after Epic Outage in Europe > “This is probably the worst-case scenario that could have happened to us.” > OVH [...] is planning to shut down and disassemble two of the three data centers on its campus in Strasbourg, France, following a power outage that brought down the entire campus Friday, causing prolonged disruption to customer applications that lasted throughout the day and well into the evening. https://www.datacenterknowledge.com/uptime/ovh-disassemble-container-data-centers-after-epic-outage-europe https://www.datacenterknowledge.com/uptime/ovh-disassemble-c...
- Macha 6y ago
  It sounds like the plan was to shutdown 1 and 4, the latter of which happened and the former which did not.
  - Scoundreller 6y ago
    It’s confusing as they say they’re shutting down 2 of the 3, but then there’s 4...

AdamJacobMuller 6y ago

> Fire destroyed SBG2 This is crazy. SBG2 was HUGE and if this isn't a translation error on the part of Octave (which I could understand given the stress and ESL) I have a hard time fathoming what kind of fire could destroy a whole facility with nearly 1000 racks of equipment spread out across separated halls. I'm really hoping "destroyed" here means "we lost all power and network core and there's smoke/fire/physical damage to SOME of that" I can't even fathom a worst-case scenario of a transformer explosion (which does occur and I've seen the aftermath of) having this big of an impact. Datacenters are built to contain and mitigate these kinds of issues. Fire breaks, dry-pipe sprinkler systems and fire-extinguishing gas systems are all designed to prevent a fire from becoming large-scale. Really glad nobody was hurt. OVH is gonna have a bad time cleaning all this up.

qbasic_forever 6y ago
Just guessing, but maybe a fire suppression system going off could wipe out all the machines? The couple datacenters I've been inside were small, old and used halon gas which wasn't supposed to destroy the machines. No idea how it works in big places these days.
- atkbrah 6y ago
  A few years back there was an incident in Sweden where noise coming from the gas based fire suppression system going off destroyed hard drives [1]. 1. https://www.theregister.com/2018/04/26/decibels_destroy_disk_drives_in_digiplex_data_centre/ https://www.theregister.com/2018/04/26/decibels_destroy_disk...
  - abrookewood 6y ago
    I've also seen a weird video (lost to time unfortunately), where someone showed that they could yell at their servers in a data centre and introduce errors (or something similar). Was very strange to see, but they had a console up clearly showing that it was having an impact.
    - 6y ago

deleted 6y ago

[deleted]