7 ms·
Google Cloud (Europe-west2) is experiencing downtime
- kulor 4y agoHas knock on effects for WPEngine which appears to use GCP (https://wpenginestatus.com/incidents/381158 https://wpenginestatus.com/incidents/381158)
- ohashi 4y agoThey aren't the only hosting company using GCP. GCP has been very aggressive targeting hosting partners. Kinsta: https://status.kinsta.com/ https://status.kinsta.com/ SiteGround: https://twitter.com/siteground/status/1270492907078914052?lang=en https://twitter.com/siteground/status/1270492907078914052?la... Both having same issue
- sz4kerto 4y agoIt's pretty bad -- I can't really find anything that works. VMs, storage, everything is hard down.
- londons_explore 4y agoIt's kinda bad, because cooling infrastructure when stretched can still cool some stuff, just less... So they ought to be able to still keep some services up for some users. They should have already practiced that as part of their emergency procedures, because they'd need to do the same is they had a severe limit on power use (eg. Due to a nationwide electricity or fuel shortage).
- capableweb 4y ago> So they ought to be able to still keep some services up for some users. Wouldn't that depend on what exactly broke here? If the cooling of the power system failed, they might end up without power for example, impacting everything.
- londons_explore 4y agoI suspect it wasn't that the cooling broke, but that the cooling couldn't keep up. Ie. The cooling could only remove XX kilowatts of heat, but that the equipment in a particular room was producing more than that.
- longwave 4y agoI have HA Cloud SQL instances in europe-west2-a that are stuck in failover state but haven't failed over. I assume this is due to the heatwave - the UK has broken temperature records today, hitting over 40°C (104°F) in London for the first time. Oracle Cloud is suffering similar outages due to cooling infrastructure issues: https://ocistatus.oraclecloud.com/ https://ocistatus.oraclecloud.com/
- dabeeeenster 4y agoAWS London had an outage last Sunday too.
- CraigRood 4y agoI’ve just lost my evening due to the Oracle Cloud, not their fault to be fair, and availability was good, but our apps weren’t happy with the performance.
- justinholmes 4y agoSummary: Cooling related failure in one of our buildings that hosts zone europe-west2-a for region europe-west2. The other zones are not impacted
- Hamuko 4y agoI guess it's worth pointing out that europe-west2 is in London, which is currently at +35°C and was as high as +40°C at one point earlier today.
- justinholmes 4y agoProbably better to point out to move to Finland (europe-north1) as it's only 18 degrees.
- mcs_ 4y agoIf everyone here in the south follows this advice., Finland will sink
- justinholmes 4y agoBelgium and Finland have better capacity than London/Zurich.
- lesuorac 4y agoContext: https://www.bing.com/search?q=guam+capsize+youtube https://www.bing.com/search?q=guam+capsize+youtube
- matthewdrussell 4y agoEquinix LD4/5/6 (Slough, Greater London) is the datacentre affected
- lol768 4y agoIt's a bit of a stretch to call europe-west2 "London" when it's Slough
- iasay 4y agoLost AWS last Sunday as well in same region due to AC failure. Hot here :)
- papito 4y agoI have to say, one thing about the U.S. is that there is AC everywhere - even up north. We have AC in elevators, on the busses - everywhere. Europe needs to start adjusting to the new normal of hot, hot summers.
- nickv 4y agoIt's a product of America's climate being more extreme on the highs and lows (a product of the jetstream). The one place that doesn't have extreme summers, the PNW, actually has very little AC too.
- Majestic121 4y agoYou say this as if it was not one of cause for hot hot summers
- saddlerustle 4y agoAir conditioning emits less CO2 than heating. Emissions would be lower if europe was hotter and everyone had AC
- kingludite 4y agoWe are under a smog alert atm. "caused by high ozone concentration" https://www.rivm.nl/en/news/smog-alert-in-force-throughout-netherlands-on-tuesday https://www.rivm.nl/en/news/smog-alert-in-force-throughout-n...
- isbvhodnvemrwvn 4y agoReversible heat pumps, not just AC.
- bzxcvbn 4y agoAir conditioning consumes a ton of energy. About 10% of all electricity consumed worldwide. https://www.iea.org/reports/the-future-of-cooling https://www.iea.org/reports/the-future-of-cooling It emits about 4% of greenhouse gases https://www.sciencedirect.com/science/article/abs/pii/S2542435122000940 https://www.sciencedirect.com/science/article/abs/pii/S25424... But the volume itself is not meaningful - the gases used by AC units are extremely powerful - up to 750 times the greenhouse effect of CO2. And of course, the AC unit itself heats the city - which is why they get much hotter than the countryside during heat waves, further exacerbating the demand for AC and making the problem even worse. It's a vicious circle. > if europe was hotter How do you suggest achieving that? Other than global warming?
- mkl95 4y agoSimple statistics can be deceiving. Just because there are no 40C+ temperature records at some location it doesn't mean that won't change next year. Southern Europe has been learning it the hard way for a while.
- upupandup 4y agoWhat's crazy to me is that a bunch of servers in AC cooled room consumes more power than entire villages in Europe suffering from this heat. Imagine being on your phone tweeting, instagram stories, whatsapp messages in the blistering weather while the databases and servers receiving and delivering data sits comfortably in AC cooled buildings. Does anybody find this contrast just eerie?
- radicaldreamer 4y agoAnd they have AC, whereas many private homes in villages have never had air conditioning.
- hunter2_ 4y agoImagine if our workloads only did the minimum calculations necessary to achieve the associated user stories, instead of constantly rebuilding from massive dependency trees and all the other inefficiencies we put up with because it takes less dev effort that way and our performance budgets are being met either way. Most places have emissions requirements for vehicles and so forth. Maybe some day we'll demand energy-efficient programming.
- londons_explore 4y agoA tax on energy effectively incentiviszes efficient use of compute. Today cloud providers deliberately site their facilities to make use of the cheapest tax-free electricity, which is effectively tax dodging.
- upupandup 4y agoI would take this further and find out what is the most energy intensive computing task across these DCs and attack those first. This is a huge unknown for me as I have no statistics on what activities result in the most energy usage in these DCs but if I have to guess, it is probably GPU/Database. I would also tax personal data. Current regulatory framework strangely assumes that personal data has no value yet entire companies are built around monetizing and claiming it as theirs. Those companies are also the biggest consumers of these DCs. Even by taxing personal data that is being used by companies, you could realize significant energy savings as they have to discern between what they need and what they dont. Asking for someones driver license to verify their account? Tax them yearly. Suddenly companies that dont really need it find themselves searching for more energy efficient solutions since their tax dollars depend on this activity. Unfortunately, we may not see this happen for quite some time.
- npstr 4y agoNice, another entry for this month's https://outage.bingo/ https://outage.bingo/
- nemothekid 4y agoLooks like I got a bingo on my first try: https://outage.bingo/?month=9&seed=10970 https://outage.bingo/?month=9&seed=10970
- remram 4y agoNote that you can use alphanumerical seeds too, for example your HN username https://outage.bingo/?month=9&seed=nemothekid https://outage.bingo/?month=9&seed=nemothekid
- meibo 4y agoAtlassian should absolutely be on this, feels like every morning I sit down at my desk and get a 504 from hosted JIRA.
- madboston 4y agoCan't believe reddit hasn't had outage. They sure go down randomly.. Maybe that's micro outage that this doesn't consider?
- myroon5 4y agoluckily not one of the largest regions: https://github.com/patmyron/cloud/#ip-addresses-per-region https://github.com/patmyron/cloud/#ip-addresses-per-region