6 ms·

Millions of websites offline after fire at French cloud services firm

60 points by jamespwilliams 6y ago

Bluestein 6y ago
This kind of thing serves me as a reminder of how "physical" our "digital" really is ...
- capableweb 6y ago
  Yes but no. Sure, millions of websites offline, most likely websites that run on single servers, in one data center. Hardly "our" digital reality is stored there, but more like personal artifacts or smaller businesses. Our digital reality lives much further than that, and is what makes up the web. It's not run on one server, sometimes in a magical cloud of instances, living in harmony.
  - Bluestein 6y ago
    > sometimes in a magical cloud of instances, living in harmony. That's part of the point here, I think: They are far from magical and, at some level, they are "physical" ... Let's call it a "gentle" reminder, then :)
tmfi 6y ago
See current discussion: https://news.ycombinator.com/item?id=26407323 https://news.ycombinator.com/item?id=26407323
janandonly 6y ago
"Millions of websites".... I have not noticed even one going off-line, but I guess France has it's own internet, a bit like North-Korea or China? Anyways, how hard is it to spin up another VPS and relocate your website there?
- scomp 6y ago
  OVH Definitely hosts a lot of seedboxes.
- chillfox 6y ago
  The article mentions that it’s website’s from across Europe that were hosted there, not just France sites. Also every country has a lot of local websites, think government and small to medium local businesses.
  - Fordec 6y ago
    Also, because of the Europe focus, a disproportionate number of those affected won't be in English. They will be in French, German, Italian etc. Part of HN's bias is at times because of the US focus, namely Silicon Valley there is quite a information bubble about what actually goes on in the world of nearly 8B people compared to the "mere" 300M in the US. Starting to lose count of the number of people claiming today they have never even heard of OVH despite it's scale. When I did my European education, the number 1 provider used in university was OVH. If millions of sites are down, and you haven't been impacted by it today, guess what, you are in that bubble. I for one know that a number of Wordpress plugins are currently borked and Lichess was affected.
    - kreelman 6y ago
      Interesting point about the info bubble. Hope people had backup/replication plans. The expensive but good side of this might be that OVH will look carefully at their setup and prevent future versions of fires like this and customers will check over their backup/replication plans. It'd be interesting (though probably impossible to do) to find out how many people didn't have plans setup for this kind of thing.
- pmontra 6y ago
  I'm not French but I have 6 websites on a server at OVH. Luckily they are in another data center. I didn't remember where they are so I was thinking about how to resurrect them from backups while I was typing the URL to check if they were not burned down.
Scoundreller 6y ago
Meh on the websites offline. What about the seedboxes??? The private VPNs? The FTPs? The ownclouds? Anything big would have had some redundancy, but the little guys didn’t.
- nathancahill 6y ago
  You just made me panic and check my seedbox. Still there.
  - devwastaken 6y ago
    What setup do you use? I had deluge-web in a container but it would corrupt the files for some reason.
    - namibj 6y ago
      rtorrent (with rutorrent) scales better, and unless you're using the deluge-gtk client, seems generally preferrable. Oh, and the core isn't in (relatively) slow Python.
    - nathancahill 6y ago
      transmission-daemon with the web GUI.
- viraptor 6y ago
  > The private VPNs? Those are stateless, just open one up in a new place. > The ownclouds? That's one of the few things that should literally refuse to run unless you prove you have backups on a different network. (not technically possible to verify of course) Without it, you will lose data at some point.
holografix 6y ago
> “There was no immediate explanation provided for the blaze, which erupted just two days after the French cloud computing firm kicked off plans for an initial public offering.“ Very interesting.
- lucb1e 6y ago
  The simplest explanation (Occam's razor) is that the events are unrelated. Speculation on that, and then citing and further spreading the idea around that maybe there are people against OVH, seems pointless (perhaps even harmful to the company) unless there are some signs of arson or people unhappy with the announcements or anything.
  - bobthebuilders 6y ago
    Occam's razor - someone got mad due to IPO and torched the place Simpler than whatever you're impling a
    - InitialLastName 6y ago
      Is it simpler? Are fires set by angry people more common than fires that happen by neglect, stupidity or equipment failure?
    - LockAndLol 6y ago
      That's not simpler because it throws up more questions. Who would want that, how did they get access, how did they plan it do quickly, how they know to target this specific one, etc. Simple is: fires happen in buildings due to failures. There was a nonhuman failure like in millions of other cases. No need for a conspiracy. Otherwise, with your logic, god explains everything because "it's simple"
  - someperson 6y ago
    As others mention OVHCloud does host VPS's and seedboxes, which have many legitimate users but can be used for crime especially when VPS providers accept Bitcoin and other difficult to track currencies. It was only a few days ago that it was been announced there is an unprecedented hack in progress of Microsoft Exchange / Outlook Web App that is reportedly being launched by the HAFNIUM hacking group in China using VPSs hosted around the world. I'm not saying arson is involved, but the possibility of arson needs to be impartially investigated. Admittedly, the falsifiability of the theory is also difficult, because even tracing back the fire to an electrical fault doesn't immediately preclude it from being a deliberate act. [1] https://krebsonsecurity.com/2021/03/a-basic-timeline-of-the-exchange-mass-hack/ https://krebsonsecurity.com/2021/03/a-basic-timeline-of-the-...
deleted 6y ago
[deleted]
Barrin92 6y ago
Says a lot about the robustness and degree of centralization of the internet that one data-center going up in flames takes a non-trivial chunk of French domains offline, including government websites. It reminds me of that story recently about the hack on that water recycling plant in the US that was only barely caught and could have poisoned a city. A lot of infrastructure really operates in barely secure fashion.
AnonsLadder 6y ago
>There was no immediate explanation provided for the blaze, which erupted just two days after the French cloud computing firm kicked off plans for an initial public offering. That pretty much sums up why the fire started

Bluestein 6y ago

This kind of thing serves me as a reminder of how "physical" our "digital" really is ...

capableweb 6y ago
Yes but no. Sure, millions of websites offline, most likely websites that run on single servers, in one data center. Hardly "our" digital reality is stored there, but more like personal artifacts or smaller businesses. Our digital reality lives much further than that, and is what makes up the web. It's not run on one server, sometimes in a magical cloud of instances, living in harmony.
- Bluestein 6y ago
  > sometimes in a magical cloud of instances, living in harmony. That's part of the point here, I think: They are far from magical and, at some level, they are "physical" ... Let's call it a "gentle" reminder, then :)

tmfi 6y ago

See current discussion: https://news.ycombinator.com/item?id=26407323 https://news.ycombinator.com/item?id=26407323

janandonly 6y ago

"Millions of websites".... I have not noticed even one going off-line, but I guess France has it's own internet, a bit like North-Korea or China? Anyways, how hard is it to spin up another VPS and relocate your website there?

scomp 6y ago
OVH Definitely hosts a lot of seedboxes.
chillfox 6y ago
The article mentions that it’s website’s from across Europe that were hosted there, not just France sites. Also every country has a lot of local websites, think government and small to medium local businesses.
- Fordec 6y ago
  Also, because of the Europe focus, a disproportionate number of those affected won't be in English. They will be in French, German, Italian etc. Part of HN's bias is at times because of the US focus, namely Silicon Valley there is quite a information bubble about what actually goes on in the world of nearly 8B people compared to the "mere" 300M in the US. Starting to lose count of the number of people claiming today they have never even heard of OVH despite it's scale. When I did my European education, the number 1 provider used in university was OVH. If millions of sites are down, and you haven't been impacted by it today, guess what, you are in that bubble. I for one know that a number of Wordpress plugins are currently borked and Lichess was affected.
  - kreelman 6y ago
    Interesting point about the info bubble. Hope people had backup/replication plans. The expensive but good side of this might be that OVH will look carefully at their setup and prevent future versions of fires like this and customers will check over their backup/replication plans. It'd be interesting (though probably impossible to do) to find out how many people didn't have plans setup for this kind of thing.
pmontra 6y ago
I'm not French but I have 6 websites on a server at OVH. Luckily they are in another data center. I didn't remember where they are so I was thinking about how to resurrect them from backups while I was typing the URL to check if they were not burned down.

Scoundreller 6y ago

Meh on the websites offline. What about the seedboxes??? The private VPNs? The FTPs? The ownclouds? Anything big would have had some redundancy, but the little guys didn’t.

nathancahill 6y ago
You just made me panic and check my seedbox. Still there.
- devwastaken 6y ago
  What setup do you use? I had deluge-web in a container but it would corrupt the files for some reason.
  - namibj 6y ago
    rtorrent (with rutorrent) scales better, and unless you're using the deluge-gtk client, seems generally preferrable. Oh, and the core isn't in (relatively) slow Python.
  - nathancahill 6y ago
    transmission-daemon with the web GUI.
viraptor 6y ago
> The private VPNs? Those are stateless, just open one up in a new place. > The ownclouds? That's one of the few things that should literally refuse to run unless you prove you have backups on a different network. (not technically possible to verify of course) Without it, you will lose data at some point.

holografix 6y ago

> “There was no immediate explanation provided for the blaze, which erupted just two days after the French cloud computing firm kicked off plans for an initial public offering.“ Very interesting.

lucb1e 6y ago
The simplest explanation (Occam's razor) is that the events are unrelated. Speculation on that, and then citing and further spreading the idea around that maybe there are people against OVH, seems pointless (perhaps even harmful to the company) unless there are some signs of arson or people unhappy with the announcements or anything.
- bobthebuilders 6y ago
  Occam's razor - someone got mad due to IPO and torched the place Simpler than whatever you're impling a
  - InitialLastName 6y ago
    Is it simpler? Are fires set by angry people more common than fires that happen by neglect, stupidity or equipment failure?
  - LockAndLol 6y ago
    That's not simpler because it throws up more questions. Who would want that, how did they get access, how did they plan it do quickly, how they know to target this specific one, etc. Simple is: fires happen in buildings due to failures. There was a nonhuman failure like in millions of other cases. No need for a conspiracy. Otherwise, with your logic, god explains everything because "it's simple"
- someperson 6y ago
  As others mention OVHCloud does host VPS's and seedboxes, which have many legitimate users but can be used for crime especially when VPS providers accept Bitcoin and other difficult to track currencies. It was only a few days ago that it was been announced there is an unprecedented hack in progress of Microsoft Exchange / Outlook Web App that is reportedly being launched by the HAFNIUM hacking group in China using VPSs hosted around the world. I'm not saying arson is involved, but the possibility of arson needs to be impartially investigated. Admittedly, the falsifiability of the theory is also difficult, because even tracing back the fire to an electrical fault doesn't immediately preclude it from being a deliberate act. [1] https://krebsonsecurity.com/2021/03/a-basic-timeline-of-the-exchange-mass-hack/ https://krebsonsecurity.com/2021/03/a-basic-timeline-of-the-...

deleted 6y ago

[deleted]

Barrin92 6y ago

Says a lot about the robustness and degree of centralization of the internet that one data-center going up in flames takes a non-trivial chunk of French domains offline, including government websites. It reminds me of that story recently about the hack on that water recycling plant in the US that was only barely caught and could have poisoned a city. A lot of infrastructure really operates in barely secure fashion.

AnonsLadder 6y ago

>There was no immediate explanation provided for the blaze, which erupted just two days after the French cloud computing firm kicked off plans for an initial public offering. That pretty much sums up why the fire started