6 ms·
WhatsApp Down in Some Regions
- tcarn 6y agoPhew, it's not just me. Beyond me why more of these big tech companies can't build more redundancies for their services.
- justapassenger 6y agoIt's not just about adding redundancies. Redundancies don't protect you against bugs, and they're itself very complex, so they introduce more opportunity for errors. Even with best redundancy, you'll have incidents from time to time.
- capableweb 6y agoIt doesn't help either that Facebook made three separate services that should have nothing to do with each other, talk to each other and all route their traffic through the same infrastructure. What they did was purposefully remove redundancy they had, in order to be able to track people more (in order to profit more) and possibly scale easier. Doing nothing would have been easier but yet they still did it.
- justapassenger 6y agoIf you think that maintaining 3 completely separate stacks for 3 different services within the same company is making all of them more reliable, I don't think you ever worked on a big scale services.
- capableweb 6y agoNo, I'm hinting at that have one company running these three services in the first place is wrong. Should be three independent companies as they are really three different services, but lord knows governments does nothing to prevent monopolies these days.
- bananaface 6y agoFacebook took $85 billion in revenue last year and has thousands of developers on the payroll. They could absolutely maintain 3 separate services. Bear in mind WhatsApp used to maintain a userbase of 200 million with 50 total employees (not even just developers).
- capableweb 6y ago> They can absolutely maintain 3 different services Lol, I'd believe you when they did purchased the service but the countless amount of times it went down since then obviously proves that they cannot even maintain the services when they are folded together on the same infrastructure. It was a long time ago Facebook employed the best of the best. Seems like it's mostly average developers and infrastructure people there now just trying to hold up the house of cards they built.
- bananaface 6y agoHah, to be clear that is what I meant - the idea that Facebook doesn't have the resources is ludicrous, the problem runs deeper.
- sodality2 6y agoThe scale of facebook is beyond what you probably imagine it to be.
- cranekam 6y agoThese companies build for redundancy. Not in the sense of "we need 10 servers, let's have 10 more for redundancy" (that gets expensive when you have millions of hosts) but by scaling out across multiple regions/DCs/clusters/etc such that there is enough slack in the system to absorb failure of 1 or 2 resource units (DCs, fiber, whatever). Also, widespread outages like this are seldom the result of insufficient capacity. They are almost always a perfect storm of several failures within systems that are individually build to handle adverse conditions. An example might be a bug within a task scheduling system that inadvertently scales down some critical service which in turn leads to something else failing to reach consensus or read configuration or who knows what. The point is that each of these components is designed and built to handle failure but something the holes in the cheese line up and the whole thing fails. In this case since IG, WA and FB were affected it's reasonable to guess the failure was in some shared component like load balancing or task placement, though (as hinted at above) the origin of the fault is not necessarily in that component directly.
- WanderPanda 6y agoIIRC there was a situation where google built a circular dependency in their own cloud services and they had massive issues bootstrapping the systems again after they went haywire
- csjr 6y agoCan confirm. Instagram is also down here.
- etimberg 6y agoInstagram also seems down, so likely something generic affecting faceebook's infrastructure
- moinnadeem 6y agoInstagram is down for me (located in the midwest). Reports "5xx server error"
- whimsicalism 6y agoBut not FB blue, interestingly enough for me.
- AugurCognito 6y agoWhatsapp is reporting same error here(India) https://web.whatsapp.com/status.json https://web.whatsapp.com/status.json
- neom 6y agoCurious if anyone knows why they would use xx instead of the number?
- cratermoon 6y agoThe HTTP response says 503, so the text is just generic for any 5xx error.
- neom 6y agoHow do you know the HTTP response was 503? (sorry for the silly questions, I'm not a software engineer)
- cratermoon 6y agoI looked at the headers returned from the server using developer tools. 503 is "Service Unavailable" and you often see it when a proxy can't reach the backend server.
- francis-io 6y agoDown for me in the UK for the past 30 mins.
- luxpir 6y agoSame. Quick, SMS everyone a Signal link!
- capableweb 6y agoYeah, lets replace this centralized service that goes down sometimes with another centralized service that hasn't started going down sometimes yet! OR, get people on Matrix or similar so we can actually stop jumping ship when our centralized dear service stops acting in our best interest.
- luxpir 6y agoHa, I hear you. Also a fan of Matrix, but it's so not mainstream usable yet. Tried to get non tech friends on there yet?
- capableweb 6y agoWhat you mean "friends"?
- luxpir 6y agoPretty much. My main preference for Signal is that it's a charitable organisation. That and a few other bits. Phone number IDs are on the way out hopefully. The dream would be some kind of federated layer, but the charitable org thing is a good start for me.
- kitkat_new 6y agodefine mainstream usable
- Spivak 6y ago
- Karawebnetwork 6y agoInstagram, WhatsApp and Messenger are all down for me.
- qwertox 6y agoSounds like paradise.
- uncledave 6y agoThis is the paradise I live in every day. Never used it. I have however had to spent the last hour being asked when it will be back by three teenagers and a wife, none of whom appreciate it is nothing whatsoever to do with me. Grr.
- Karawebnetwork 6y agoIt would be, but that made me fall into the HN rabbit hole. One hour later, I know more about the hacks used in Duke Nukem 3D but my tasks are still uncompleted.
- zakember 6y agoWait till you find out about YouTube
- mrpinc 6y agoMassive global outage https://t.co/boWJaMxDG0?amp=1 https://t.co/boWJaMxDG0?amp=1
- dkokelley 6y agoThere's a broader outage somewhere, and I've had degraded service on Google Meet today. Thought it was my speed, but speedtest.net is down too! https://downforeveryoneorjustme.com/speedtest.net https://downforeveryoneorjustme.com/speedtest.net
- qwertox 6y agoI agree, the landing page of this post ( https://xn--allestrungen-9ib.de https://xn--allestrungen-9ib.de ) shows that almost all services are affected.
- deleted 6y ago[deleted]
- bukhtarkhan 6y agothis is exactly why we need 1000000 different messaging apps lol
- totalZero 6y agothat, or just one app that uses multiple pathways and protocols to get the message across
- hans1729 6y ago>(xn--allestrungen-9ib.de) Jesus Christ HN, can you please use a sane charset
- capableweb 6y agoEh, what's the problem really? A) we who regularly use those characters already know how it looks when websites don't encode stuff properly B) the crowd on HN most likely know what it means anyways, and can ignore it without loosing their mind
- hans1729 6y ago>(xn--allestrungen-9ib.de) >whats the problem just no
- capableweb 6y agoYou want to explain the problem or just shit on something you don't understand? I thought we were all here to learn and discuss with good intentions, but seems you're not...
- hans1729 6y agoCan you not make this an "blablabla ad hominem blabla"? It is absolutely obvious why this is terrible behaviour: the url that's previewed simply doesn't match the one you'd type in the address bar. The encoding is messed up despite Umlaute being part of utf8, and if you're parsing HN with your own code, you're in trouble because you need to account for encoding, which simply shouldn't be a thing in 2021. "HN should be about ideas, so instead I'll make this about you" is not just wasting every readers energy, it's blatant projection and beyond embarassing. PLEASE don't do this here.
- cratermoon 6y agoThat's a good point, so maybe it's a HN display issue.
- marcodiego 6y agoDown in Brazil too together with instagram and facebook. More food for conspiracy theorists...
- Daishiman 6y agoYou mean all products owned by the same company, most likely hosted in the same data centers and with the same infra?
- deleted 6y ago[deleted]
- marcodiego 6y agoConspiracy theorists can see whatever they want mixed in the noise.
- narrowtux 6y agoAny day now they will find out which company is behind all three apps.
- acruns 6y agoThey are in the same dc? Worst design evaaah
- TedShiller 6y agogood
- rootsudo 6y agoJust five eyes.
- mbit8 6y agoits down in germany too
- deleted 6y ago[deleted]
- sparkling 6y agoFB up but very slow, Instagram and Whatsapp down Germany
- fritex 6y agoInstagram too!
- deleted 6y ago[deleted]
- shever 6y agoCompletely down in Ireland too.
- adrianb 6y agoDowndetector [1] shows some problems with other websites including Twitter and Microsoft as well. Doesn't seem like a cloud provider being down though. [1] https://downdetector.com/ https://downdetector.com/
- justin4all 6y agoDecentralized services like Matrix - anyone? No, let's continue relying on a single centralized service for the whole world.
- sneak 6y agoThis isn't the panacea you think it is. For example, on iOS all server notifications for an app must be proxied via Apple and have to come from the app developer's push certificate. This means that even with Element (the matrix client) all of your notifications when your phone is locked are going from your server to the centralized dev push system run by the Element devs, then to another centralized system run by Apple. Either could have an outage, although I think APNS (the apple side) has close to 100% uptime since launch, which is quite a feat.
- saghm 6y agoThis comes from from someone who enjoys the convenience of centralized services, but that honestly sounds to me like an argument for decentralization of mobile notifications too, not a refutation of the idea of decentralized messaging.
- sneak 6y ago"We" can't decentralize notifications on iOS, only Apple can. It's not in their interest to do so (battery life, platform brand control via censorship), so it's probably a moot point. It's better to choose a fight you can win.
- toyg 6y agoThis comes from someone typing this on an iPhone, but that sounds like an argument to drop this goddamn walled garden/prison. I’m literally sick of it.
- sneak 6y agoYou also can't use an iPhone without it constantly sending usage information to a half dozen different teams/departments at Apple, even if you've opted out of analytics, so there's that, too. I think I've bought my last iPhone, as well, after having had all of them.
- mimarko 6y agoWhat, in some regions??? Better to say in almost all regions
- mimarko 6y agoWhat, in some region? Better to say in almost all regions :)
- forlorn 6y agoSame for instagram
- cratermoon 6y agoOh wow, I don't think I've seen a real IDNA/Punycode domain in the wild before.
- sneak 6y agoYou probably have and didn't realize it.
- DyslexicAtheist 6y agoquite rare on HN yes. there are loads depending on the region. if you have no use for them it's a good idea to block them with dnscrypt-proxy[1] or nfq[2] to avoid getting bitten by homograph attacks. [1] https://github.com/DNSCrypt/dnscrypt-proxy https://github.com/DNSCrypt/dnscrypt-proxy [2] https://gitlab.com/jbauernberger/nfq/ https://gitlab.com/jbauernberger/nfq/
- phreack 6y agoEven Telegram is suffering from a slightly degraded service, likely from the load of everyone jumping to it at once as a backup IM.
- tebbers 6y agoDown in London.
- sodality2 6y agoSlowly instagram is coming back? seems like progress is being made edit: push notifs came through. inbox not loading however edit 2: seems to be fully restored. all my homies are responding to messages
- Shorel 6y agoDon't panic. It is just the usual NSA backdoor upgrade. Last week it was Azure. Next week Apple services will be upgraded to the latest version. Stay tuned!
- rvz 6y agoAnd no. The same people won't flock to Signal after all of this, even though it is much more friendlier than Element. On the other hand, at least Element is decentralised thanks to Matrix, but unfortunately suffers from a naming dilemma and is unfriendlier than the rest of the chat alternatives. We need an alternative that is a mixture of both: Open source, Decentralised (Likely uses Matrix underneath) and is extremely user friendly and competes on security and features.
- tuwtuwtuwtuw 6y agoNo, you need an alternative which is used by some substantial portion of people.
- abhinav22 6y agoDown in Dubai too
- iKevinShah 6y agoAnd they are back up (at least for me, India). I hope it was not a push on Friday evening or worse "after EoD".
- toyg 6y agoProbably a push on Friday morning on the US West Coast...
- metahost 6y agoDown in Spain too.
- deleted 6y ago[deleted]
- AbraKdabra 6y agoIt's already up in Argentina.
- cblconfederate 6y agoLet's not get overexcited ... they will probably come back up again.
- refraincomment 6y agoHow will people be happy today?
- zarq 6y agoThe site presents me with a pop-up: > When you proceed to access our site, the companies listed in the Cookie Consent Tool will use cookies and other technologies. This is further explained in our Cookie Notice. None of those are links, and the only button is "Agree and access site". How do I find the Cookie Consent Tool and the Cookie Notice, without having to click "Agree"?
- MauranKilom 6y agoI get the same here in Germany (if I turn off uBlock Origin). It's not GDPR complaint, plain and simple.
- qwertox 6y ago> and other technologies Do they need to specify these other technologies, which they are and how they are being used?
- ncallaway 6y agoI personally inspected the page to delete the popup from the DOM without clicking "agree". I found their privacy page here: https://xn--allestrungen-9ib.de/privacy.html https://xn--allestrungen-9ib.de/privacy.html. Interestingly, that has a link to a cookie policy that links to speedtest.net's cookie policy (https://www.speedtest.net/de/about/cookie-policy https://www.speedtest.net/de/about/cookie-policy). I think downdetector is offered by Ookla, so that probably is their cookie policy? speedtest.net's cookie policy leaves something to be desired. In the english translation it has: > If you prefer that we do not collect information that will help us determine which advertisements we are best serving you, click this icon. But I don't see an icon there, or anything clickable. In case it's a mistranslation here's the original german: > Wenn Sie es vorziehen, dass wir keine Informationen erfassen, mit deren Hilfe wir ermitteln können, welche Werbung wir Ihnen am besten anzeigen, klicken Sie auf dieses Symbol. Based on that, I have no idea how you can manage your cookie preferences to express a desire not to be tracked. It seems to be an absolutely flagrant violation of GDPR, and I would encourage a German citizen to bring a case to the German data protection authorities.