17 ms·
I suppose this is at the top of HN because of Reddit having an outage and Hacker News being slow. But neither Reddit nor HN use Cloudflare. And if you look at t
by jgrahamc 3y ago
I suppose this is at the top of HN because of Reddit having an outage and Hacker News being slow. But neither Reddit nor HN use Cloudflare. And if you look at the status page there this isn't our core CDN offering, it's certain products that are affected.
- capableweb 3y agoI suppose this is at the top of HN as CloudFlare is quickly become a centralized point of failure for big parts of the internet, even without counting the CDN, so large swaths are affected regardless if it's just a "minor" outage or not. Hence it's interesting enough to land at the top.
- robertlagrant 3y agoI wouldn't say it's a centralised point of failure. Many individual sites choose to use it, so it is used by big parts of the internet, but that doesn't make it centralised. Maybe a single point of failure.
- efdee 3y agoIt's impossible for a single point of failure not to be centralised. Otherwise it wouldn't be a _single_ point of failure.
- robertlagrant 3y agoOf course it is. If a million websites are built with Flask then a bug in Flask affects all of them. If a million websites individually decide to use Cloudflare then Cloudflare affects them all. But that doesn't mean Cloudflare is built in a centralised way (I assume it isn't), nor that there's something about the internet that is centralised around Cloudflare. Rather, people are choosing to include it as a dependency in their stack.
- efdee 3y agoUsing a piece of software is not the same as using an online service. If all those websites are using Flask, that would be the equivalent of centralising on Flask. The opposite of that would be many Flask-compatible yet unrelated other frameworks being used in parallel. A bug in Flask would not affect those not using that very codebase. The centralisation people are speaking of here is the amount of people all putting their eggs in Cloudflare's basket.
- robertlagrant 3y ago> If all those websites are using Flask, that would be the equivalent of centralising on Flask I agree on the equivalence between this and Cloudflare use, but not that this should be described as centralisation, which is a particularly potent word on the open web. Popularity isn't the same as centralisation. Each website could rewrite to remove Flask if they wanted. Another example: if lots of people watch Squid Game, that doesn't indicate a centralisation of television programmes. No options are removed. People are just choosing to do a similar thing, but not in a way that centralises anything.
- p4bl0 3y agoIn your example Flask would be a single point of failure and could indeed be called decentralized. For example websites can individually patch their Flask installation and come back up one by one, without depending on anyone else (at least once Flask is fixed). Here with Cloudflare, a single entity is responsible for the fix and will more or less fix the failure for every sites that uses it at the same time, by fixing it on their side. And website individually cannot do anything on their own. So I would argue it makes sense to call that centralized, at least from a structural/operational perspective. It is at the very least a form of contraction of the network.
- robertlagrant 3y agoFrom an operational perspective it's just a supplier like any other. Lots of suppliers are involved in a business's website. That doesn't make it centralised, though? > It is at the very least a form of contraction of the network. I think it's that at most. No one has to use them, as they accelerate / enhance open protocols. That is the least lock-in one could hope for, so they don't contract anything in a negative way. Contrast with, say, Etsy/Shopify, who actively try to replace the open space with closed ones.
- pixl97 3y ago>It's impossible for a single point of failure not to be centralised. Is a monoculture centralized? It doesn't have to be, and yet it can all fail due to a single fault.
- activiation 3y agoThere are so many websites using it that it is basically the same as being centralized.
- sgt 3y agoSo many sites using Flask across the web is centralized?
- activiation 3y agoIf most sites were using Flask and forced to do broken updates, it would be a centralized problem.
- r4indeer 3y agoYou're comparing apples to oranges. Flask is software of which all instances run independent of each other while Cloudflare is SaaS where all its sites depend on the same service. If Cloudflare goes down, so does a significant portion of the web. Flask cannot have an outage like Cloudflare. All they can do is push a faulty update, but even then you can rollback or stay on the old version.
- tetha 3y ago> If Cloudflare goes down, so does a significant portion of the web. Flask cannot have an outage like Cloudflare. All they can do is push a faulty update, but even then you can rollback or stay on the old version. This is what was on the tip of my tongue. I'd argue you're missing the portion of control in this whole discussion, and how much process you can place in front of a component change. If I have a flask dependency, I have a lot of control over this dependency. If flask screws up badly, I have many options: I can update. I can not update. I can downgrade. In fact, I could fork flask internally and fix it on my own and either be a good citizen and open up a PR, or I could be something else. I can test all of this in any number of environments before it hits a customer, and even more stages before it hits all customers. With Cloudflare - or any number of Hosters as well, like AWS, Azure, Google Cloud, I have very little control. If I use Cloudflare as a CDN and Cloudflare goes down, I might not have the capacities at my upstream server to handle the load from all my customers, so I am down as long as Cloudflare is down. I wouldn't have the footprint available necessary to replace AWS in our own private DCs, even if we pooled all spare capacities - and then I'd still have to find a way to exfil our data from a downed AWS. (Which, yes, we have, but it'll take long hours) And no matter how much I test, if my hoster fucks up, I'm immediately fucked as well, no matter what processes I might have. The only process around this would be provider independence, which is really expensive and a lot of effort even if you just have a luke-warm standby.
- NicoJuicy 3y agoThere's also the perception issue that if the host fails, it most likely isn't cloudflare although cloudflare gives a warning. Note: experienced it from the first row, cloud had an issue in a region with SSL...
- deleted 3y ago[deleted]
- cyanydeez 3y agoBUT THE INTERNET IS SUPPOSE TO ROUTE AROUND TTHIS STUFF. Remember when crypto promised a decentralized utopia of distributed systems that any interference wouldnt work, but slowly but surely it all congregated into a couple of oligarchical companies? Surely by now the technology of the internet has matured enough that these conglomerates are going to do the same but with a bit less fraud involved.
- supriyo-biswas 3y agoThe internet does work around issues at the inter-networking layer through BGP and similar protocols, though the same resilience is sadly absent at the higher layers.
- dangerlibrary 3y agohttps://en.wikipedia.org/wiki/Border_Gateway_Protocol https://en.wikipedia.org/wiki/Border_Gateway_Protocol BGP was standardized in 1989 and has been in use since 1994. The technology of the internet has matured in many ways, and remains almost identical in many others. Sometimes Microsoft is right, and backwards compatibility is the killer feature.
- sph 3y agoWTF has crypto got to do with centralisation of the Internet? The Internet is well-decentralised, but if every Joe Blogg has to put their website behind Cloudflare, it's not the Internet's fault, let alone crypto (?). Stop putting every-bloody-thing behind Cloudflare, and the problem solves itself. I don't know whether to laugh or cry when I read of someone on HN seriously saying that they need a CDN for their personal website, or that they really need to use AWS or GCP for that matter. We tech workers have lost the plot, and it's our fault it's all in the hands of the few.
- pluto_modadic 3y agoI think they're referencing when "distributed" exchanges still went down when particular APIs or domains were unreachable. It ain't fault tolerant unless it's piracy >:D
- 3y ago
- MichaelZuo 3y agoIs there some sort of backup/failsafe mechanism for sites that use Cloudflare?
- viraptor 3y agoDepends how badly they fail. To take the full advantage of CF you need to keep your DNS with them. That means if you can't configure any changes, you can't quickly move to another provider either. Your only solution at that point is to transfer the whole domain, but that may also require CF's assistance. (Unless they don't handle your apex domain) Effectively unless they're down for more than a day, you're better off taking the hit and waiting for them to resolve everything.
- ta1243 3y ago> Effectively unless they're down for more than a day, you're better off taking the hit and waiting for them to resolve everything. lol. There's me worrying about a 2 minute downtime once every 10 years
- deleted 3y ago[deleted]
- gerdesj 3y agoWell I've managed ~53.5 years uptime with zero downtime so far 8) I'm taking the piss too. Care to explain? PS For a laugh (and I apologize for going dreadfully off topic), I asked ChatGPT a couple of questions regarding two mins and 10 years, just in case I'd missed a trick with your comment. I got two calculations within both answers that looked the same but have a factor of 10 difference in the result! https://chat.openai.com/share/7aaa80e5-1c54-4ac2-9ddb-9e488d1757bd https://chat.openai.com/share/7aaa80e5-1c54-4ac2-9ddb-9e488d... This is where it goes wrong: "Total minutes in 10 years = 60 minutes/hour * 24 hours/day * 365 days/year * 10 years = 525,600 minutes" "Total time in 10 years = 60 minutes/hour * 24 hours/day * 365 days/year * 10 years = 5,256,000 minutes" LO Calc says that =6024365*10 = 5,256,000. The worrying thing for me is that an awful lot of input training data might be badly wrong to cause this result or perhaps I've managed to excise a corner case in which case the training data is a bit too focussed in this particular regard. I suspect that arithmetic errors for these things will be awful because there are so many ways to screw up and the training data will have a lot of errors in it. Combine that with the number of subjects available and it will be a shit show.
- dahfizz 3y agoAnecdotally: op guessed right - I was only interested because of the HN and reddit outages
- BiteCode_dev 3y agoTrue. But if they fetch a lot of URLs from cloudflare and it takes 30 seconds to answer with a timeout instead of a http/200 under 20ms, then some architectural decisions that were sound under the latter case may make the whole system slow in the former one.
- lanstin 3y agoPeople have started testing network failures but so often fail to test slow network failures. All of a sudden your code has 500x more pending open sockets, both in and out, and the memory spikes and it all goes to hell. Even if the fast fail code path is indeed best effort.
- mysterydip 3y agoBasically the old "I Love Lucy" chocolate factory sketch: https://youtu.be/AnHiAWlrYQc https://youtu.be/AnHiAWlrYQc
- bombcar 3y agoI would love to see graphs showing "length of reply that eventually succeeded" - I suspect that most networks today, if you don't get a response in 5 seconds, you ain't never gonna get anything useful. In other words, I wonder if going to fail fast would help the health of the Internet more than wait forever timeouts. Might reduce DDoS effects, too.
- dylan604 3y ago>I suspect that most networks today, I guess that comes with the type of end point in that network. A typical website, absolutely, yes, I'd agree. An API end point allowing requests of large data pools that might take a few seconds to generate but yet not a total time out would be acceptable.
- BiteCode_dev 3y agoWell a proper system should reduce the timeout for the same domain, the more it gets raised, and set it back to default once stats are good again. But it's very complicated and costly to setup, so almost nobody does this.
- arek_nawo 3y agoI've been experiencing occasional slow-downs and issues on HN since last week. I would presume that it's due to more load (with WWDC last week and now Reddit black-out) but I don't really see this fully-reflected in the comment or up-vote count on the main page (except the recent Vision Pro thread).
- deleted 3y ago[deleted]
- teaearlgraycold 3y agoAn ex admin of Reddit thinks the protest broke the internal cache layer performance: https://tildes.net/~tech/163e/reddit_appears_to_be_down_during_blackout_day_1#comment-87v1 https://tildes.net/~tech/163e/reddit_appears_to_be_down_duri...
- EamonnMR 3y agoWhoa tildes are a reddit alternative? I thought it was all about, like, public access unix systems.
- DiggyJohnson 3y agoIs there a known feasible way of receiving an invite at the present time?
- teaearlgraycold 3y agoHuh? It’s not quite an alternative. It’s good though and I’d recommend joining once there’s another invite wave.
- progval 3y agotildes.net seems completely unrelated to the ones you are thinking of (the tildeverse)
- EamonnMR 3y agoAh, thanks.
- dom96 3y agoAnyone have an invite to share for Tildes?
- buildbot 3y agoWould love one too…
- senttoschool 3y agoPossibly third parties that Reddit/HN rely on are using Cloudflare?
- Aardwolf 3y agoIs HN slow because everyone who was normally posting on reddit is now commenting on HN instead?
- doxeddaily 3y agoI think so. I'm trying to move my time over here as much as possible.
- DoctorOW 3y agoSame here. Even the subreddits that are still open, I'd feel dirty reading.
- starttoaster 3y agoTime to upgrade from a t2.Micro to a t3.Medium, I suppose.
- freedomben 3y agoGiven the innaccurate and clickbaity title "Cloudflare is having Issues" (when in reality it's just a few services, not the main stuff Cloudflare is largely known for), I suspect a lot of people are upvoting and not actually following the link. A comprehensive Cloudflare issue would be huge news. The truth here is not huge news IMHO.
- k8sToGo 3y agoThe statement “Cloudflare is having issues” is true as soon as count(issues) > 0. So what’s so wrong about the title? And how do you know how many websites use R2 etc. so that you can jump to the conclusion that it’s not huge news?
- HWR_14 3y ago> The statement “Cloudflare is having issues” is true as soon as count(issues) > 0. Pedantically, it's true as soon as "count(issues) > 1"
- freedomben 3y agoExactly. The headline isn't untrue, but it is misleading. Most people hear "Cloudflare is having issues" and assume Cloudflare as a whole, and this is not an unreasonable assumption. Reductio ad absurdum can be used to demonstrate the logic flaw. Let's assume that one person using Dynamo DB got a 500 error response from a two successive API calls (so an error rate of ~4.0e-09). It would technically be true for the headline to say only "AWS is having issues." A headline like that is going to rocket to the top of HN, and it's not going to provide many people with useful information. It's also silly. There's plenty of room in the headline to instead of "Cloudflare is having issues" to say "Cloudflare R2, Stream Live, and others are having issues."
- anikom15 3y agoThis is at the top of HN because Cloudflare is not delivering a product people pay them for.
- ChrisMarshallNY 3y agoI'm downloading a large app update (MAMP Pro). It's screaming along at 47KBsec. I'll have it all in about two and a half hours. Not sure if that is related.
- jbergstroem 3y agoFYI: I didn't see any notices about disruptions for workers but were intermittently unable to use `wrangler publish`: https://i.imgur.com/LUAzoHQ.png https://i.imgur.com/LUAzoHQ.png