10 ms·
Tell HN: GitHub is down (Update: Back online now)
Getting 500 errors, multiple friends confirming: https://github.com
- mman0114 5y agoYup, confirmed here as well. Multiple team members also having issues.
- tgymnich 5y agosame
- AdamJacobMuller 5y agodown for me
- lnsp 5y agoAccording to GH Status (https://www.githubstatus.com/ https://www.githubstatus.com/), everything is fine. Gotta love functional status pages. edit: Nevermind, they just reported "degraded performance" for GitHub Actions, Issues, and Pull Requests.
- rossmohax 5y agoDidn't they have actual error rate graphs on that page back in a day?
- speedgoose 5y agoYes, and some response times too. It was actually useful.
- agomez314 5y agoi wonder how they even work? Like, do they just display a green css button instead of _actually_ doing a healthcheck?
- deleted 5y ago[deleted]
- drstewart 5y agoProbably requires manual updates. It seems like more and more places have moved to this paradigm now that status pages are tied to SLAs which are tied to money. One might call it the politicization of status pages.
- asciimike 5y agoCorrect. SLAs aren't calculated off status pages, there are far better ways of calculating it (running a query over responses, for example). Most modern SLAs are customer initiated anyways, so the customer is writing in to request this rather than automatically calculating them. The status page doesn't need to show anything for a customer to provide logs indicating a QoS less than that promised in the SLA. I don't think it's politics (maybe AWS's is, but GCP wasn't IMO), it's really a function of "in large scale software systems things are constantly failing in all sorts of ways, and it's really hard to output a meaningful automated signal that things are broken. Sure you can set up pingdom type health checks on every endpoint, but even then you're not necessarily guaranteeing that things are working properly. Source: worked at a few cloud providers, paid out a few SLA violations
- deathanatos 5y agoPoliticization, yes; I've never heard of SLAs being tied to status pages. It is like pulling teeth to get most cloud providers to credit the account when they don't meet SLA, and one always has to ask for it; heaven forbid if credits were paid out automatically when service wasn't rendered. Or you get weasel-worded out of it. I had a cloud provider deny a service credit; the SLA stated that the service was only out of SLA if it didn't return 2xx. Well, the API returned "2xx Accepted — your request is being processed", and you could use the API to query the job, and the job … never finished or made any progress at all. But the API returned 2xx the entire time, so that was "within SLA".
- mkl95 5y agoAWS do the same thing. Pretty sure someone updates it manually.
- YPCrumble 5y ago
- diogenesjunior 5y agoYou gonna sue them?
- mirekrusin 5y agoMaybe they run it from github actions?
- nefasti 5y agoFraud implies intent.
- na85 5y agoNot a lawyer, but my understanding of this issue is that it constitutes fraud if and only if the status page is a contractually- or legally-binding warrant as to the status of their services. >why is it legal in the tech industry? Tech has a history of abusing legal gray areas or else simply ignoring laws it finds inconvenient, enabled by toothless and sluggish enforcement, and using profits from lawbreaking to fund lobbying campaigns to retro-actively make things legal. For recent examples see Uber, Airbnb, Clearview, et al.
- lowercased 5y agoIt seemed like a good faith effort... Between when I was told about it (which seemed like it was very early on) to when I saw the status page change (saw it 'all clear' at first) was... 3 minutes? Yes, it's not 'real time' but they don't seem to be intentionally hiding something for hours on end. There may be some consequences for reporting on false positives too quickly as well.
- heipei 5y agoI can't imagine the status page is mentioned anywhere in the contracts. I would think that the only language that is included is that it's the customers responsibility to track and notify the vendor (Github) about any downtime, and to do that within 48 hours of the downtime having occured. And only then will the customer be eligible for compensation, probably in the form of free service credits. My personal take is that this wording is the reason you see so few public status pages in general, especially ones with automatic and minute-by-minute status history. Better to put it on the customer to have accurate monitoring in place, which most people simply won't have.
- sc90 5y agoThey have updated it.
- russian_bot 5y agocan corroborate, very down
- daveed 5y agoLooks down to me :(
- oneil 5y agoLiterally just posted a thread.. Doh! :(
- zaphod4prez 5y agoYep, having issues
- devalnor 5y agodown for me too
- lijogdfljk 5y agoYup. Not even the unauthenticated home page is showing up. Wew
- sp1rit 5y agoYes, down in Europe too :/ https://downforeveryoneorjustme.com/github https://downforeveryoneorjustme.com/github
- codingkev 5y agoYep, down since ~10mins here in central europe.
- sctgrhm 5y agoSame here
- mkl95 5y agoYup. I've been disrupted by GitHub today, and DockerHub the other day. Crude reminder that the cloud is some company's computer.
- longnguyen 5y agoLooks like only the website is down. I just used the mobile app and it works fine.
- sc90 5y agoYes, returning a 500.
- asciimike 5y agogithub.com is having issues across the site, the API including git operations (and CLI) still work. Status page is manually updated, and we're working to get it updated. EDIT: it's updated now. EDIT EDIT: github.com is back up and running, apologies for the disruption :( Source: GitHub employee
- Miner49er 5y agoWhy manually update vs automatically?
- jondwillis 5y agoThey wait for the HN thread to appear so they don't give false positives.
- JaimeThompson 5y agoI think it is so they can "hide" small outages that don't rise to the level of making the news sites so they can look better. A lot of sites do this sort of thing these days.
- asciimike 5y agoIsh? I wrote some stuff up here: https://news.ycombinator.com/item?id=30182591 https://news.ycombinator.com/item?id=30182591 When I worked at GCP it was all manually updated as well so we could add a sentence about what was actually affected. In any sufficiently large system it's hard to indicate exactly what's broken/how to work around it, so it was just easier to `/status <system> <color> <reason for status>`.
- kxrm 5y agoNot many places I have worked for allowed for this to be automatic. A lot of it was so they could provide a coherent explanation as to what the current state of internal attention was directed at vs what everyone can plainly see.
- 5y ago
- dvdhnt 5y agoRight as we were doing a big release, too :)
- romellem 5y agoIncident is up now - https://www.githubstatus.com/incidents/fz1bdbw24y81 https://www.githubstatus.com/incidents/fz1bdbw24y81 > We are investigating reports of degraded performance for GitHub Actions, Issues, and Pull Requests.
- TremendousJudge 5y agoThat's a funny way of saying "every page is returning 500"
- drewbug01 5y agoIIRC, that's the default verbiage that goes out when someone pulls the "oh shit" lever.
- xtracto 5y agoDegraded Performance? Their freaky homepage is borked: https://github.com/ https://github.com/ yields a 500 error haha.
- AdamJacobMuller 5y ago100% degraded is degraded.
- rossmohax 5y agoNegative growth rate is still growth :)
- deleted 5y ago[deleted]
- deleted 5y ago[deleted]
- leesalminen 5y agoI just refreshed a page from GH that I've had open since last night, and yup, 500. Of course I came to HN first before even visiting their own status page as HN always has an update faster than their official page.
- rvz 5y agoOh dear. Last time a serious incident happened was just 48 hours ago: [0] Now it has gotten critically worse. Is it time to use a self-hosted backup like what GNOME is using? [1] [0] https://news.ycombinator.com/item?id=30149071 https://news.ycombinator.com/item?id=30149071 [1] https://gitlab.gnome.org/GNOME https://gitlab.gnome.org/GNOME
- sdan 5y ago
- gregmfoster 5y agographite.dev (alternative frontend to github code review) still up, it appears to be just their frontend.
- caterama 5y agoTheir 500 page is delightful... More downtime, I say!
- joshstrange 5y agoI am able to push code but then I went to check the GH Action to make sure it ran and.... error pages.
- rsyring 5y agoI'd like to see a graphic of the traffic spike at times like this from CTRL-R. They even encourage it on the 500 page: "try refreshing". Edit: it's back.
- eskendar 5y ago
- pedrodelfino 5y agoIt was down for me too (in Brazil). But it is back now! :)
- nyellin 5y agoIt's like the good old days when the power went down at work and everyone came to the hallways. Good luck to the engineers at GitHub as I know how stressful it can be, but hope everyone else is enjoying a nice break and some socializing
- kitkat_new 5y agoToo bad federated GitLab still does not exist.
- filippofinke 5y agoIt's currently working in Switzerland.
- sambhu 5y agoIt's up and running now
- zmmogv 5y ago
- karussell 5y agoWorks again for me here in Germany.
- MatthiasPortzel 5y agoIt looks like it's back up now, can anyone confirm?
- sylware 5y ago