6 ms·
OVHcloud fire class action reaches 140 clients, seeks more than €10m
- sofixa 4y agoI'm not sure the lawsuit has much merit. OVH's datacenter designs are custom, widely described by themselves, and on their terms. You sign up for their services based on what they provide, they didn't promise much, and when their SLA was broken they paid up. That's how they remain so low cost, they skimp on everything they can, like Ryanair. If i can draw an analogy, it'd be like buying food from a food truck selling very cheap seafood, which follows all relevant regulations, but has a booklet describing how that seafood takes a week to get to them, and you get sick. Yes, they should do their job better, but nobody can act surprised.
- usr1106 4y agoExcept that most countries have public (state and/or local) surveillance of food safety and those not complying will be fined or can be closed. With reliability of IT services such thing probably doesn't exist in any country. Only when it comes cybersecurity they could be fined for violating GDPR. But I don't think they leaked anything, they just lost it.
- sofixa 4y agoThat's why i said, they followed all relevant regulations ( which for datacenters basically stop at fire safety). Their implementation was really bad, but compliant.
- yakak 4y agoIt will be interesting to see. If for example there were somewhat relevant fire or electrical regulations that weren't properly enforced by local government, I guess the owner or builder is still the only one liable?
- tyingq 4y ago"OVHcloud fire report: SBG2 data center had wooden ceilings, no extinguisher, and no power cut-out"[1] I suspect that opens up negligence, which may route you around legal barriers like SLAs and TOS, depending on the laws where this lawsuit is happening. And of course, you don't have to win in court to win. You can just be enough of a cost, nuisance, or PR problem to get a settlement. [1] https://www.datacenterdynamics.com/en/news/ovhcloud-fire-report-sbg2-data-center-had-wooden-ceilings-no-extinguisher-and-no-power-cut-out/ https://www.datacenterdynamics.com/en/news/ovhcloud-fire-rep...
- xxpor 4y agoDo those tactics work in European countries? The legal regimes are quite different from the US/England
- consp 4y agoAt least where I live negligence is a thing but this may vary wildly. If no reasonable person would expect you to do it that way then you can claim negligence, YMMV how much you get out of it. The problem usually is that you have to either go after the business which goes belly up as soon as you try and you still get nothing or after the owners money and claim irresponsible management which is far harder to do (and has a criminal code part).
- rtpg 4y agoNo extinguishers seems pretty surprising, does OVH not have all those certifications that surely ask for such things?
- mateo1 4y agoI would expect any large datacenter to have a halon based fire suppressant system or something similar.
- gruez 4y ago>I suspect that opens up negligence I suspect that'd only be the case if these are in contravention of building/fire codes. Is that the case? Given that the building is a datacenter, it might be subject to less regulations compared to something like an apartment.
- 908B64B197 4y ago> That's how they remain so low cost, they skimp on everything they can, like Ryanair. That's also why you go with a real cloud as soon as you raise enough funding.
- capableweb 4y agoUnless you want to be cost-effective and have the expertise, then bare-metal is the way to go. As things seem right now, companies are gonna get a lot better at being cost-effective and cloud services are not.
- bayindirh 4y agoI don't whether you're also including "build your own data center" inside bare-metal, but a data center has many moving parts. Keeping servers up and running is the easiest of all.
- Nextgrid 4y agoIf a fire happens every 20 years then I'll take my chances. It's not like the "real clouds" don't go down either.
- tgsovlerkhgsel 4y agoOr have a good DR strategy/a second cheap cloud provider ready, potentially giving you more reliability than a single expensive one (see e.g. https://news.ycombinator.com/item?id=17431609 https://news.ycombinator.com/item?id=17431609 and the horror stories posted in the comments there). Harder if you're heavily relying on the more advanced services with often provider-specific APIs, but I think if you host with OVH you're likely to be primarily renting machines.
- xuki 4y agoYeah, I was affected by another datacenter fire (WebNX Utah). I had hourly back up, everything was back and running on Digital Ocean within an hour. There are reasons to use AWS/Google Cloud/Azure but reliability is not it.
- bayindirh 4y agoI'm not sure that OVH is meeting all criteria for a safe building containing a lot of wood for any use, let alone for data center use. Let's see what will come out of this.
- dylan604 4y agoThat's so crazy to me that it could even be allowed after an inspection for a certificate of occupancy. I worked for a company that leased space in a very corporate filled with laywers/finance type of tennants, but this company, a video post house, was very much not either of those. The building kept requiring expense after expense (partly in hopes of making it so expensive they'd choose to go somewhere else). Part of these expenses were that all cabling had to be run through conduit including CAT5, phone cabling, 75ohm coax video cables. In the room that doubled as the machine room and small data center, they required smoke detectors every 6 feet. And these knuckleheads got to build one in what sounds like a wooden hut with no electrical cut off? Sheesh!
- dx034 4y agoBecause no one lives in a datacenter and there was only water around the building.
- bayindirh 4y agoA datacenter, esp. when you store the UPS batteries near, is a ticking dirty bomb. Many of the systems have rechargeable batteries (UPS batteries & RAID controllers' battery backup), a lot of rare earth metals which release toxic fumes when burnt, etc. It's extremely hard to extinguish a full-blown electrical fire, backed by UPS batteries and other nasty chemicals and supported by a multi-megawatt power connection. You don't need to harm people directly. The fumes' effects are enough of an hazard for people already.
- dx034 4y agoThat's why the fire at the OVH dc took so long to extinguish. They had to stay away for hours and get boats to provide more water. Sure there were toxic gases, but I'm not aware of people being harmed by it, especially with no one living anywhere close.
- tinus_hn 4y agoYou can’t run a food truck and not guarantee your food doesn’t make people sick, that’s not even allowed in the Land of the Free.
- SnowHill9902 4y agoYou can never guarantee that your food doesn’t make people sick. You can and must follow all regulations, guidelines and best-practices but that’s on a best-effort basis.
- wmf 4y agoDid they really document that there's no isolation between SBG1 and SBG2?
- dx034 4y agoI'd never trust multiple datacenters on one site. If they're not at least a mile from each other, I treat them as one datacenter. OVH has 4 locations in France alone, there's no reason not to use different cities instead of just SBG1+2.
- deleted 4y ago[deleted]
- closewith 4y ago> That's how they remain so low cost, they skimp on everything they can, like Ryanair. Ryanair has arguably the best safety record of any airline in the world (with no fatal accidents or incidents in their history, and an order of magnitude more flights/PAX PA than the other contenders such as Hawaiian), so they definitely don't skimp on everything.
- derefr 4y ago"Everything they can" isn't the same as "everything." "Everything they can" means "everything they can get away with as a long-term-viable business strategy." For an airline, safety is not one of those things. For OVH, though, the "safety" of their servers (i.e. availability/durability) is one of those things they can skimp on — because unlike for people, for servers, "safety" is just another fungible thing to optimize as part of OpEx. One of the fundamental tenets of IT operations is that hardware can just fail, datacenters can just catch fire(!), backhoes can just cut cables, etc. — and so businesses that care, architect their stacks to be resilient to these failure modes, using multi-AZ replication and so forth. And if everybody who actually cares is already doing that... then it really doesn't matter if your DC goes up in smoke. (And also, to these same companies, low cost can have a "safety" all its own. Rather than lowering OpEx, you can instead see the calculus as: for the price of one vertically-scaled AWS instance, you can get four equivalent OVH boxes in different regions, and replicate between them. Which one of those options is "safer"?) Mind you, if anything got burnt up that had real SLAs on it — for example, if they were offering an DBaaS service out of that datacenter — they'd be in trouble. But some leased bare-metal boxes? Nah.
- throw_m239339 4y ago> Ryanair has arguably the best safety record of any airline in the world (with no fatal accidents or incidents in their history, and an order of magnitude more flights/PAX PA than the other contenders such as Hawaiian), so they definitely don't skimp on everything. Ryanair is also a relatively young company compared to other air companies, and flights are mostly within Europe, not inter continents. I took a Ryanair plane once, that plane was a complete wreck and I thought I'd die. No more. And all things considered they aren't even that cheap since they often fly from remote airports in the middle of nowhere, so you need to get there first, wasting money and time.
- throwoutway 4y agoWhat’s funny is I’ve seen several HN posts disown hyperscale cloud providers and encourage others to use OVH based simply on cost, without any analysis of what they lose in the process Another poster pointed out that the data center had wooden ceilings! That’s insane!
- sascha_sl 4y agoWhat most of these posts also say is that you need a good strategy for handling catastrophic failure of single data centers. Even if you deploy your workload 3x redundant with offsite backups, you're likely still paying less than on the 3 big clouds in traffic alone.
- throwoutway 4y agoI’ve never seen a single OVH fan say that in their recommendation here
- sascha_sl 4y agoI have (about Hetzner)
- dx034 4y agoIf your server goes down it doesn't matter if it's because of a faulty host or because of a dc fire. You need redundancy either way. People affected by this fire had likely all assets in one data center, which I also wouldn't recommend with AWS & Co.
- sofixa 4y agoIt's slightly different, because AWS have AZs. Each region is composed of at least 3 AZs, each one being at least a datacenter, at a distance from one another, but the region is usable as a single entity. An OVH DC is separate, but can be next to other DCs in an area ( e.g. the Strasbourg DCs were all next to each other) which is OK because they don't sell any AZ or regional features.
- deleted 4y ago[deleted]
- spaniard89277 4y agoI wonder how cheap a VPS provider can go. In Spain none of the companies can match OVH, but datacenters in Spain are pretty old regular ones, even the small ones like NixVal.
- spiffytech 4y agohttps://lowendbox.com/ https://lowendbox.com/ is the go-to place for finding the most shoestring VPS available, and they can come very cheap, either in absolute dollars or in compute-per-dollar.
- Havoc 4y agoOr lately https://lowendspirit.com/ https://lowendspirit.com/ - split off from the above. I find the providers there slightly less shady. Though again fly by night is entirely possible so backup accordingly
- 0des 4y agoThanks, I didn't know about this one. Where can I read more about the split? derp: https://lowendspirit.com/discussion/2317/a-year-in-les-reflections-thoughts-my-history-and-the-future-for-lowendspirit/p1 https://lowendspirit.com/discussion/2317/a-year-in-les-refle...
- Havoc 4y agoYep though current admin is Mason who seems to be running the forum as a no-ads passion project. For now anyway
- dx034 4y agoHetzner can match OVH. To match their prices you need scale to build your own datacenters and servers. Small providers can't do that, they need to colocate and that'll usually be more expensive than owning your dc.
- 4y ago
- 2OEH8eoCRo0 4y ago
- teaearlgraycold 4y agoHold up - are people using OVH for production/business work?
- speedgoose 4y agoYes, many people deployed a lot in OVH. Their reputation wasn’t perfect but much better before the fire (and the loss of the managed backups drama). They were also big before AWS or Azure arrived in Europe.
- throwaway744678 4y agoObviously yes, why wouldn't they? I know for a fact that I do. Yes, I was affected by the fire, and had to pull the backups and rebuild the whole stack - we got a few hours of downtime. And yet I stayed with OVH, the main reason being that, even taking into account this unfortunate event, you get a great value for your money. Of course, some businesses can't afford any downtime: in this case, whichever provider one chooses, they have to go multi-cloud, or at least distribute their setup on multiple regions. I have been burned (figuratively!) in the past all the same with big names cloud provider... Shit just happens, be prepared!
- markvdb 4y ago> Obviously yes, why wouldn't they? Because OVH's negligence transpires more widely than just in DC fires. I've had my primary business domain name almost expire twice due to OVH's incompetence. I should have run away the first time. I finally did the second time. Never again, thankyouverymuch. Some low cost providers do offer acceptable quality. OVH was (is?) not one of those.
- the8472 4y agoWhat is acceptable varies. Some things are very low-stakes. Even if your domain-name is important you could manage it with another registrar and still use OVH for their cheap metal.
- dx034 4y ago
- rgbrenner 4y agoNone of these clients owned these servers… so the class action is only for lost data. But none of these people had backups? They weren’t worried about a failed hard drive or a power outage causing data corruption? Or a bad stick of ram? Or cosmic rays? Or software bugs? Or hardware bugs? Or hackers? Or ransomware? Or user error? Or … Seriously.. weren’t these people just waiting for something bad to happen? And now that something bad did happen, they want someone else to pay for it.. because they wouldn’t pay for the cheapest of measures to protect their property they claim was so valuable. Have you looked at how cheap hard drives are today?