5 ms·
Cloudflare was down
https://blog.cloudflare.com/cloudflare-service-outage-june-12-2025/ https://blog.cloudflare.com/cloudflare-service-outage-june-1...
- aranchelk 1y agoSeems to be affecting functionality of their "Verify you are human" dialogs as well as Workers.
- clairegraham 1y agoYep, KV is broken too. Any worker that depends on KV is throwing exceptions. I was able to get into the dash, but it's very slow. Error rates started to go up significantly around 18:00 UTC. Edit: The CF status page has acknowledged it's a broad outage across many services: https://www.cloudflarestatus.com/incidents/25r9t0vz99rp https://www.cloudflarestatus.com/incidents/25r9t0vz99rp
- aranchelk 1y agoAfter many tries I also got into the dashboard, but it's not that usable, constant error pop-ups.
- deleted 1y ago[deleted]
- bgwalter 1y agoIt does. Another question is why do we get these dialogues always from Cloudflare and never from Akamai in the first place?
- bgwalter 1y agoDownvoting this comment and flagging the submission does not address the serious issue. These verification dialogues make the Internet unusable.
- perching_aix 1y agoNor does venting about it in unrelated threads, or asserting your opinion as fact.
- bgwalter 1y agoI complained in the apnews.com thread, because the apnews.com verification, which is annoying by itself, did not work at all this time. That is hardly unrelated.
- scubbo 1y agoIt's not much of a reach to go from "discussion about impact on human-verification dialogs" to 'discussion about human-verification dialog policy". This isn't an incident-management channel, it's a discussion forum - tangents are fine!
- iimblack 1y agoThey updated the incident noting that it's not just authentication affected.
- pier25 1y agoWorkers KV has been down for like +30mins. This is impacting us seriously. Their API is down too. Amazing that something can impact their whole infrastructure like this given how much redundance they have.
- stri8ted 1y agoThe same is true for Google.
- nijave 1y ago>can impact their whole infrastructure CDN and WAF seem to be working fine. I think CF rushed a lot of newer services out without the reliability some of their older/core services enjoy
- kenhwang 1y agoFrom their incident page (https://www.cloudflarestatus.com/incidents/25r9t0vz99rp https://www.cloudflarestatus.com/incidents/25r9t0vz99rp): > Cloudflare’s critical Workers KV service went offline due to an outage of a 3rd party service that is a key dependency. I bet that 3rd party service is GCP. I would be pretty pissed if I were a CF customer that used Workers KV for redundancy because it was heavily marketed as running on CF data centers.
- jerrygoyal 1y agoGCP is also down https://news.ycombinator.com/item?id=44260810 https://news.ycombinator.com/item?id=44260810
- ipsum2 1y agoOdd coincidence. Wonder if Cloudflare uses GCP?
- ikiris 1y agoIt's likely their auth infra based on what the Google outage is
- devmor 1y agoWhat do you mean by this? The Google outage is a widespread outage of most GCP services.
- pageandrew 1y agoGoogle is claiming the root cause is with some of their central IAM services, which would have a cascading effect to the rest of their services.
- devmor 1y agoWhere did you see this information? Was it on a social media channel? I do see the IAM services in the list of affected services in the incident report.
- tom1337 1y agocheck https://status.cloud.google.com/incidents/ow5i3PPK96RduMcb1SsW https://status.cloud.google.com/incidents/ow5i3PPK96RduMcb1S... > Multiple GCP products are experiencing impact due to Identity and Access Management Service Issue
- ineedaj0b 1y agosolar flare?
- deleted 1y ago[deleted]
- CoopaTroopa 1y agoNo, Cloudflare.
- pier25 1y agoThey've changed the title to "Broad Cloudflare service outages"
- paxys 1y agoLet me guess, someone pushed out a bad BGP config?
- CSMastermind 1y agoFor an outage this large and widespread that would have to be the main culprit.
- deleted 1y ago[deleted]
- neo_doom 1y agoYeah this is going to be a problem. I haven't seen an issue this widespread across so many services in a while.
- tete 1y agoSeems to be semi regular now that everyone puts all their eggs in only a few baskets.
- solardev 1y agoI gotta say, it's kinda nice when that happens... work just kinda pauses for everyone, from providers to customers. It kinda feels like a national holiday, and everyone downstream from the affected cloud can just kinda sit back and relax cuz there's nothing they can do anyway except wait. When it's your own outage, it's all-hands-on-deck panic mode. When it's half the internet down, it's no longer your problem, lol
- prauscher 1y agoI guess it depends on what your company's acceptable level of downtime is. If you're like Cloudflare (who handled this well), you take this as a sign to build fault tolerance around your 3rd party providers. If your application is mission-critical, downtime is anything but a holiday.
- joduplessis 1y agoHopefully they also publish the prompt that did this.
- tough 1y agoi was thinking about this too
- daxfohl 1y agoThey should make the AI lead the postmortem.
- deleted 1y ago[deleted]
- vsgherzi 1y agoThey're just moving fast and breaking things 100x faster. Who cares what code does just vibe it all away /s
- artursapek 1y agolmao
- b0a04gl 1y ago[dead]
- vimwizard 1y agoproxy seems available in general, must just be local to workers because only one of my sites going thru ZT tunnel with identity access rules is affected
- ourmandave 1y agoIs it coincidence that there's a Scheduled Maintenance in Tokyo for 18:00 UTC in progress, and the problems started at 18:19 UTC?
- perching_aix 1y agoGuess we'll find out from the postmortem. Always the silver lining with these, get to learn from and enjoy a good writeup.
- solarmist 1y agoDo these get posted publicly?
- perching_aix 1y ago> Do these get posted publicly? Yes.
- solardev 1y agoYeah. Cloudflare writes some of the best ones in the industry, and they're very enjoyable to read: https://blog.cloudflare.com/tag/post-mortem/ https://blog.cloudflare.com/tag/post-mortem/ I really do appreciate the transparency and ownership that comes with these. We all fuck up, but a lot of companies would rather hide their mistakes than own up to them. Cloudflare's approach makes me trust them more.
- bhaney 1y agoProbably
- jonfw 1y agoThere is always scheduled maintenance on that page, so that's not much of a signal in my experience
- alexcroox 1y agoUnrelated, they have a few services that rely on GCP which is down. Still, I imagine the people working on the maintenance for Tokyo turned white during that job worried it was caused by them...
- koliber 1y agohttps://downdetector.com/ https://downdetector.com/ is showing outages at many major companies including Google, CloudFlare, AWS and more. Word on the street is that there are large BGP routing issues behind all of this.
- cogman10 1y agoWould make sense. I think the last time I saw this sort of thing it was BGP causing a bunch of traffic to route through Iran or china IIRC.
- koliber 1y agoI vaguely recall that incident. But it did not feel like it affected this many services. At the same time I have not noticed anything being down firsthand. I am in Europe.
- cogman10 1y agoHere's the case [1]. Looks like they targeted a single /24 so that's likely why it wasn't a bigger issue. [1] https://bishopfox.com/blog/bgp-hijacking-technical-post-mortem https://bishopfox.com/blog/bgp-hijacking-technical-post-mort...
- NooneAtAll3 1y agoso this is related to Israel's escalation that everyone is expecting?
- CoopaTroopa 1y agoThe Pentagon Pizza Report has been having a lot of activity the past 24 hours. Maybe just a coincidence
- nijave 1y agoThere was also an older instance with China https://www.cyberdefensemagazine.com/experts-detailed-how-china-telecom-used-bgp-hijacking-to-redirect-traffic-worldwide/ https://www.cyberdefensemagazine.com/experts-detailed-how-ch...
- deleted 1y ago[deleted]
- deleted 1y ago[deleted]
- deleted 1y ago[deleted]
- deleted 1y ago[deleted]
- tete 1y agoBig blog post about how they saved the internet upcoming. ;) Currently down, but reference: https://blog.cloudflare.com/the-ddos-that-almost-broke-the-internet/ https://blog.cloudflare.com/the-ddos-that-almost-broke-the-i...
- claudex 1y ago> Cloudflare’s critical Workers KV service went offline due to an outage of a 3rd party service that is a key dependency. So they depend on GCP for (some of) their services
- reimertz 1y agowrote a similar comment - good to know for the future.
- QuantumNoodle 1y agoIf that is true, and there is no other BGP shenanigans, then I suspect this dependency will not be around for long
- beastman82 1y agoMy WAG is it comprises 95% of the company infrastructure
- IX-103 1y agoI heard that it was a "mandatory dependency" to mitigate "insider risk" or something. There's no way it's going anywhere. Odds are they'll just enforce even slower rollouts "to catch things early".
- SahAssar 1y agoWAG = wild-assed guess?
- tempaccount420 1y agoMaybe beastman82 is wagging his tail?
- pizzafeelsright 1y agoceo just said not for long
- 1y ago
- deleted 1y ago[deleted]
- pier25 1y agoOur Workers apps are up again edit: It works in the US but EU customers are still reporting our services as down. edit: EU customers are reporting ok
- poorman 1y agoCan’t wait to read this post-mortem. Seems odd that a Google Cloud outage would bring down Cloudflare services.
- sidcool 1y agoCloudflare's lava lamps are dimming.
- PeterStuer 1y agoSo both Cloudflare authentication as well as Google's identity systems suffered major dowtime yesterday. Are there technical dependecies between these?
- tom1337 1y agoCloudflare doesn't say this directly but in their blog they've written > The cause of this outage was due to a failure in the underlying storage infrastructure used by our Workers KV service, which is a critical dependency for many Cloudflare products and relied upon for configuration, authentication and asset delivery across the affected services. Part of this infrastructure is backed by a third-party cloud provider, which experienced an outage today and directly impacted availability of our KV service.