5 ms·
This is not good. One major outage? Something exceptional. Several outages in a short time? As someone thats worked in operations, I have empathy; there are so
by pm90 10mo ago
This is not good. One major outage? Something exceptional. Several outages in a short time? As someone thats worked in operations, I have empathy; there are so many “temp havks” that are put in place for incidents. but the rest of the world won’t… they’re gonna suffer a massive reputation loss if this goes on as long as the last one.
- karmakurtisaani 10mo agoProbably fired a lot of their best people in the past few years and replaced it with AI. They have a de-facto monopoly, so we'll just accept it and wait patiently until they fix the problem. You know, business as usual in the grift economy.
- rvz 10mo agoThe "AI agents" are on holiday when an outage like this happens.
- 5d41402abc4b 10mo ago>They have a de-facto monopoly On what? There are lots of CDN providers out there.
- mvdtnz 10mo agoThis didn't happen at all. You're just completely making shit up.
- karmakurtisaani 10mo agoYeah I am a bit.
- PlotCitizen 10mo agoThis is a good reminder for everyone to reconsider making all of their websites depend on a single centralized point of failure. There are many alternatives to the different services which Cloudflare offers.
- koakuma-chan 10mo agoMy Cloudflare Pages website works fine.
- coffeebeqn 10mo agoWe just love to merge the internet into single points of failure
- phatfish 10mo agoThis is just how free markets work, on the internet with no "physical" limitations it is simply accelerated. Left alone corporations to rival governments emerge, which are completely unaccountable. At least there is some accountability of governments to the people, depending on your flavour of government.
- mschuster91 10mo agono one loves the need for CDNs other than maybe video streaming services. the problem is, below a certain scale you can't operate anything on the internet these days without hiding behind a WAF/CDN combo... with the cut-off mark being "we can afford a 24/7 ops team". even if you run a small niche forum no one cares about, all it takes is one disgruntled donghead that you ban to ruin the fun - ddos attacks are cheap and easy to get these days. and on top of that comes the shodan skiddie crowd. some 0day pops up, chances are high someone WILL try it out in less than 60 minutes. hell, look into any web server log, the amount of blind guessing attacks (e.g. /wp-admin/..., /system/login, /user/login) or path traversal attempts is insane. CDN/WAFs are a natural and inevitable outcome of our governments and regulatory agencies not giving a shit about internet security and punishing bad actors.
- berkes 10mo agoBut the nature of a CDN and most other products CF offers, is central by nature. If you switch from CF to the next CF competitor, you've not improved this dependency. The alternative here, is complex or even non-existing. Complex would be some system that allows you to hotswap a CDN, or to have fallback DDOS protection services, or to build you own in-house. Which, IMO, is the worst to do if your business is elsewhere. If you sell, say, petfood online, the dependency-risk that comes with a vendor like CF, quite certainly is less than the investment needed- and risk associted with- building a DDOS protection or CDN on your own; all investment that's not directed to selling more pet-food or get higher margins at doing so.
- rvz 10mo agoWe are now seeing which companies do not consider the third party risk of single point of failures in systems they do not control as part of their infrastructure and what their contingency plan is. It turns out so far, there isn't one. Other than contacting the CEO of Cloudflare rather than switching on a temporary mitigation measure to ensure minimal downtime. Therefore, many engineers at affected companies would have failed their own systems design interviews.
- cryptonym 10mo agoSometimes it's not worth it. Your plan is just to accept you'll be off for a day or two, while you switch to a competitor.
- rvz 10mo agoCan't say that when it is a time critical service such as hospitals, banks, financial institutions or air-traffic control services.
- cryptonym 10mo agoOnly a fool would build an architecture for critical air-traffic with Cloudflare as a SPoF.
- rvz 10mo agoMy point still stands. Having no backup / contingency plan even if any third party system goes down on a time critical service means that you want to risk another disaster around the corner. In those industries, accepting to wait for them for a "day or two" is not only unacceptable, it isn't even an option.
- creamyhorror 10mo agoIf there's a fitting competitor worth switching to. Plus most people don't get blamed when AWS (or to a lesser extent Cloudflare) goes down, since everyone knows more than half the world is down, so there's not an urgent motivation to develop multi-vendor capability.
- berkes 10mo agoAt least this warrants a good review of anyone's dependency on cloudflare. If it turns out that this was really just random bad luck, it shouldn't affect their reputation (if humans were rational, that is...) But if it is what many people seem to imply, that this is the outcome of internal problems/cuttings/restructuring/profit-increase etc, then I truly very much hope it affects their reputation. But I'm afraid it won't. Just like Microsoft continues to push out software, that, compared to competitors, is unstable, insecure, frustrating to use, lacks features, etc, without it harming their reputation or even bottomlines too much. I'm afraid Cloudflare has a de-facto monopoly (technically: big moat) and can get away with offering poorer quality, for increasing pricing by now.
- coffeebeqn 10mo agoVibe infrastructure
- rvz 10mo agoSo that is what the best case definition of what "Vibe Engineering" is.
- MrAureliusR 10mo agowell that's the thing, such a huge number of companies route all their traffic through Cloudflare. This is at least partially because for a long time, there was no other company that could really do what Cloudflare does, especially not at the scales they do. As much as I despise Cloudflare as a company, their blog posts about stopping attacks and such are extremely interesting. The amount of bandwidth their network can absorb is jaw-dropping. I've said to many people/friends that use Cloudflare to look elsewhere. When such a huge percentage of the internet flows through a single provider, and when that provider offers a service that allows them to decrypt all your traffic (if you let them install HTTPS certs for you), not only is that a hugely juicy target for nation-states but the company itself has too much power. But again, what other companies can offer the insane amount of protection they can?
- zelphirkalt 10mo agoMicrosoft's reputation couldn't be much lower at this point, that's their trick. The issue is the uninformed masses being led to use Windows when they buy a computer. They don't even know how much better a system could work, and so they accept whatever is shoved down their throats.
- pyuser583 10mo agoLots of big sites are down
- belter 10mo agoThis will be another post-mortem of...config file messed...did not catch...promise to be doing better next....We are sorry. They problem is architectural.
- lucyjojo 10mo agocloudflare is a huge system in active development. it will randomly fail. there is no way it cannot. there is a point where the cost to not fail simply becomes too high.
- jcmfernandes 10mo agoAbsolutely. I wouldn’t be surprised if they turned the heat up a little after the last incident. The result? Even more incidents.
- bluerooibos 10mo agoI'm quite sure the reputational damage has already been done. How do they not have better isolation of these issues, or redundancy of some sort?
- brandensilva 10mo agoThe seed has been planted. It will take awhile for others to fill the void. Still the big players see this as an opportunity to steal market share if Cloudflare cannot live up to their reputation.
- wooque 10mo ago2 days ago they had outage that affected Europe, Cloudflare seems to be going down the drain. I removed it for my personal sites.