6 ms·
GitHub was down
- deleted 2y ago[deleted]
- big-green-man 2y agoIn the last 5 minutes too, wow.
- rafamvc 2y agodown for me
- elashri 2y agoI think it maybe global but at least it is down in US-East (for sure).
- Gasu16 2y ago[dead]
- upbeat_general 2y agoDown for me as well. Thought my SSH agent was broken.
- deleted 2y ago[deleted]
- darth_aardvark 2y agoSure is.
- red_Seashell_32 2y agoStatus page, like usually all green -> https://www.githubstatus.com/ https://www.githubstatus.com/
- martins_irbe 2y agowould be better if it's down too :D
- Ygg2 2y agoIt's yellow/red now.
- twp 2y agohttps://www.githubstatus.com/ https://www.githubstatus.com/ reports no problems, but it's clearly down for a lot of people (including me).
- tabbott 2y agoIt is kinda amazing how consistently status pages show everything fine during a total outage. It's not that hard to connect a status page to end-to-end monitoring statistics...
- frabjoused 2y agoIt's simply too soon for the status page to report the anomaly, is my guess. It's been down for 4 minutes.
- thih9 2y ago4 minutes is a long time for something that could have been an automated check. For the record, the status page eventually got updated - around 7 minutes after this submission was created.
- cortesoft 2y agoThere is always going to be SOME delay between the outage and the status page, although 5 minutes is probably enough time where it should be updated
- thund 2y agoafter several minutes the status page is still showing all is fine. For a service like GH, anything more than 30 secs is unacceptable
- x86a 2y agoThat is very unrealistic. Infrastructure monitoring at that scale won't even be collecting metrics at that interval. And simple HTTP monitoring would be too flappy for a public status page.
- deleted 2y ago[deleted]
- sebmellen 2y agoI've never seen an outage this big. Even the homepage doesn't load. We've had recurrent issues with Actions not running, but this seems a lot bigger. The status page says all is well, though: https://www.githubstatus.com/ https://www.githubstatus.com/. Hilarious.
- TacticalCoder 2y ago> I've never seen an outage this big. I remember a time when systems would boast about their "five nines" uptime. It was before anything "cloud" appeared.
- ergocoder 2y agoI wonder why the status just doesn't ping github.com for 200. That seems easy to do.
- tinyhitman 2y agodelaying SLA
- sebmellen 2y agoThis is at least a multi-million dollar payout (if they admit to it). All GitHub Pages say > We're having a really bad day. > The Unicorns have taken over. We're doing our best to get them under control and get GitHub back up and running.
- deleted 2y ago[deleted]
- runnr_az 2y agodown in phx az
- bitbasher 2y agoThe timing is pretty uncanny. I just deployed a github page and had a DNS issue because I configured it wrong. I hit "check again" and github went down. Hope I don't appear in the incident report.
- deleted 2y ago[deleted]
- RIMR 2y agoThis will all clear up when it finishes checking your DNS configuration I bet.
- red_Seashell_32 2y agoWait. You use github pages for something or actually work on it?
- bitbasher 2y agoI use it for something. I had a github page that was public, but it was made private and the DNS config was removed. Fast forward to today. I made the private repo public again and forced a deploy of the page without making a new commit. It said the DNS config was incomplete, so I tweaked it and hit "check again" and github went down. Probably unrelated, but the timing was spooky.
- paledot 2y agoYour domain isn't `null.example.com` or something, is it?
- dang 2y agoSorry for the offtopicness - would you mind emailing hn@ycombinator.com so I can check in with you about a couple things regarding https://news.ycombinator.com/item?id=41221186 https://news.ycombinator.com/item?id=41221186?
- OutOfHere 2y ago
- java-man 2y agoThe status page should have a button "Report Outage".
- martins_irbe 2y agojust wanted to do one last final commit :D good timing
- waveBidder 2y agoI wonder how much it would've taken to keep github running without Microsoft buying them and running them into the ground like this.
- gray_-_wolf 2y agoThe macho unicorn is kinda dope though.
- Flop7331 2y agoSo that's why I haven't heard back on my applications
- j3s 2y agofor everyone complaining about the status page - status pages are normally operated by hand by design, and will rarely reflect things in real-time. give the poor github ops folks a second to get things moving.
- manquer 2y agoMost status page products integrate to monitoring tools like Datadog[1], large teams like github would have it automated. You ideally do not want to be making a decision on whether to update a status page or not during the first few minutes of an incident, bean counters inevitably tend to get involved to delay/not declare downtime if there is a manual process. It is more likely the threshold is kept a bit higher than couple of minutes to reducing false positives rates, not because of manual updates. [1] https://www.atlassian.com/software/statuspage/integrations https://www.atlassian.com/software/statuspage/integrations
- xyzzy_plugh 2y agoNah, _most_ status pages are hand updated to avoid false positives, and to avoid alerting customers when they otherwise would not have noticed. Very, very few organizations go out of their way to _tell_ customers they failed to meet their SLA proactively. GitHub's SLA remedy clause even stipulates that the customer is responsible for tracking availability, which GitHub will then work to confirm.
- xelamonster 2y agoLove that HN is a better status page for dev services than most companies can manage to provide. Knew I'd find it here but on the front page within 3 minutes is impressive.
- deleted 2y ago[deleted]
- kinduff 2y agoReminds me of a repository I once found when searching for Prometheus exporters. It stated this but with Twitter; it will monitor latest tweets searching for a custom word combo and raise a server alert when found. I found it hilarious. Will post the source once GitHub is back on.
- tmvnty 2y agoHN need to publish their secrets on how they rarely goes down!
- peterlk 2y agoBased on what I've read in the past, I believe the secret is simplicity. Simplicity scales
- RIMR 2y agoExcept that the weird HN algo just saw 187 upvotes in 15 minutes, and dropped this thread to the second page...
- leeoniya 2y agoit knows we're all in a voting ring called Github Users
- deleted 2y ago[deleted]
- rvz 2y agoI guess when GitHub goes down, it is somehow strangely tolerated for years even after the acquisition and goes down more times than Twitter. When the latter encounters a speed-bump, just like the 'interview' with Trump, it's global news because a Mr Elon Musk owns it. Both seem to be doing too much all at once. But really it is worse with Github if this is what Microsoft stewardship is incidents every, single week and each month guaranteed for years. Anyways. #hugops for the GitHub team.
- dlahoda 2y agoI see more and more people use less Github, but some other git solutions. I am afraid to think what to do when GitHub is down for hours (need to learn maillists?). Another reason is that MS may be in phase when it will ask to pay for using GitHub just for reads (rate limiter).
- curtis3389 2y agoI recently looked into using Git in a decentralized way. It's actually pretty easy! When you would usually create a PR, you use `git format-patch` to create a patch file and send that to whoever is going to merge it. They create a branch and use `git am` to apply the patch to it, review the changes, and merge it to main. It is nice that git supports multiple remotes, though. It feels good to know that `git push` might not work for my project right now, but I know `git push srht` will get the code off of my laptop.
- outworlder 2y agoI used to work at a company with very draconian policies. Whenever I needed to update some code on a public GitHub repository, I would just push to a remote that was a flash drive. Plug it in my machine at home, pull from that remote, push to origin. I also had to setup a bidirectional mirror back when bandwidth to some countries was restricted. We would push and pull as normal, and a job would keep our mainline in sync. It is sad that most organizations forget that git is distributed by nature. We often get requests to setup VPNs and all sorts of craziness, when a simple push to a bare mirror would suffice. You don't even need anything running, other than SSH.
- __float 2y agoDraconian policies...but not security ones? Why were USB drives not blocked?
- Gormo 2y ago> I recently looked into using Git in a decentralized way. It's actually pretty easy! Well, that's how it was designed to work! The whole point of Git is that it's a distributed version control system, and doesn't need to rely on a centralized source of truth.
- Khaine 2y agoDown in Australia as well
- ralusek 2y agoDown under?
- suyash 2y agoRIP to all those who host their websites on GitHub pages :(
- jacooper 2y agoFirst time in a while that pages goes down with github itself, it's usually separate from the main site.
- bogwog 2y agoAnybody who publishes an app on the Google Play store and hosts their privacy policy on Github pages may have their app taken down because Google's bots won't be able to verify it exists. That happened to me a while back with an app listing that was almost 10 years old because the server I was hosting the policy on went down. Ironically, I switched it to Github pages so it wouldn't happen again.
- testergrave 2y agoI have my client's app policy on GitHub. I have to check if anything happened to it. The websites are working fine
- iNate2000 2y agoI checked, and my pages-based site was down, but it is back up now.
- xaviergmail 2y ago[flagged]
- sweca 2y agoThis will have a fun post mortem
- EADDRINUSE 2y agoThings seems to be ack'ed: ``` Investigating - We are investigating reports of degraded availability for Actions, Pages and Pull Requests Aug 14, 2024 - 23:11 UTC ```
- damiankennedy 2y agoInvestigating - We are investigating reports of degraded availability for Actions, Pages and Pull Requests Aug 14, 2024 - 23:11 UTC
- damiankennedy 2y agoUpdate - Pages is experiencing degraded availability. We are continuing to investigate. Aug 14, 2024 - 23:12 UTC
- damiankennedy 2y agoUpdate - Copilot is experiencing degraded availability. We are continuing to investigate. Aug 14, 2024 - 23:13 UTC
- damiankennedy 2y agoUpdate - We are investigating reports of issues with GitHub.com and GitHub API. We will continue to keep users updated on progress towards mitigation. Aug 14, 2024 - 23:16 UTC EDIT: The reply link is no longer available. Update - Packages is experiencing degraded availability. We are continuing to investigate. Aug 14, 2024 - 23:18 UTC
- damiankennedy 2y agoThe reply link is now available? Update - Issues is experiencing degraded availability. We are continuing to investigate. Aug 14, 2024 - 23:19 UTC Update - Git Operations is experiencing degraded availability. We are continuing to investigate. Aug 14, 2024 - 23:19 UTC EDIT: The reply link is no longer available again.
- damiankennedy 2y agoThe reply link is now available again? Everything is red now. Nearly lunch time in New Zealand.
- rvz 2y agoAnd so goes all your packages, private repositories, pages, AI intern copilot bot and Github Actions; and soon your AI models once you host them there - all being unavailable and going down with GitHub. Time to consider self-hosting like the old days instead of this weekly chaos at GitHub.
- vivgui 2y agoDown in the Dominican Republic as well, was just trying to commit and end my day
- jftuga 2y agoDown Detector link: https://downdetector.com/status/github/ https://downdetector.com/status/github/
- bijant 2y agotime to go to bed then. Wasn't getting any useful work done any more anyways...
- brashad 2y ago[flagged]
- dataspun 2y agoThank goodness for HN status reports.
- maximilianroos 2y agoEven GitHub-hosted Pages are down — https://prql-lang.org/ https://prql-lang.org/ is also a unicorn
- Dibby053 2y agoFeels bad to have one's job interrupted. Looking on the bright side this is the excuse I needed to check out Radicle...
- dataspun 2y agochecked out radicle: doesn't do windows
- flkiwi 2y agoMe: I think I'll update nixos* Nix: barfs voluminous errors I've never seen before Me: whaaaat the farrrrk * nixos updates are pulled from a github repo
- dlahoda 2y agoYeah, need more caches and backup git links (including local clones). Also they had IPFS attempts, but not finished.
- arianvanp 2y agoThey're pulled from our CDN by default. Only if you use experimental flakes is GitHub in the loop. And even if GitHub isn't down you can't pull nixpkgs more than twice per hour without running into rate limits and get your IP banned. Don't rely on GitHub for critical infrastructure. https://github.com/NixOS/nix/issues/6975 https://github.com/NixOS/nix/issues/6975
- bangaladore 2y agoAnd other services like copilot... [error] [auth] Response content-type is text/html; charset=utf-8 (status=503)
- dataspun 2y agodown! unicorn! fatal: unable to access: 502 cli, web, and iOS app :-/
- fabbari 2y agoNothing useful on the status page: https://www.githubstatus.com/ https://www.githubstatus.com/
- reneberlin 2y agoYes. https://www.githubstatus.com/ https://www.githubstatus.com/
- deleted 2y ago[deleted]
- startages 2y agoIt's 00:16, just about to go to bed, I ran `git push` and it's not working. Check Github, says it's down, I think it's only me, maybe I'm blocked, Github can't be down. Come here to check and it's down for everyone, such a relief.
- testergrave 2y agoReally it was a relief. Same case for me. Now I have no energy to push my code. Tomorrow maybe
- rhabarba 2y agoI love how the same people who try to drag me towards using Git are the only people who seem to have serious problems working on their code when a website goes down.
- waveBidder 2y agoGit is not the same thing as github. It's designed to be decentralized, even if it isn't getting used that way atm
- rhabarba 2y agoI am quite familiar with the basic functionality of Git. However, I am always amused by how it works in practice.
- deleted 2y ago[deleted]
- stogot 2y agoWonder if they’ve had worse uptime after moving to Azure
- BenjiWiebe 2y agoWhat'll it be this time? DNS or BGP?
- abhishekjha 2y agoYes : https://github.com/psf/black/ https://github.com/psf/black/ is 502
- shishirraven 2y agoYes, github is not working right now
- cropcirclbureau 2y agoThere goes Pages, there goes the CDN for release artifacts, there goes any package manager hosting repositories on GitHub. Is this outage just contained to github or is it an Azure outage?
- cwilby 2y agoIt looks to be contained to just GitHub, azure service page shows no outages at this time.
- higgins 2y agoYes https://www.githubstatus.com/ https://www.githubstatus.com/
- keyle 2y agoYes it went down about 5 mins ago, I got the angry unicorn. Since then the status page is increasingly red. Seeing it all kind of went sideways at the same time, my money is on the typical load balancer config rollout snafu. "As part of a routine configuration deploym..." [splat]
- _andrei_ 2y agoYes, everything is down https://www.githubstatus.com/ https://www.githubstatus.com/
- uknownuser 2y agoYep
- josvdwest 2y agoWould this explain why "npm install next-sanity" doesn't work properly, or am I hitting a user error?
- count_countules 2y agocould be if the package is hosted on github
- Gasu16 2y ago[dead]
- weystrom 2y agoA reminder of how centralized and dependent the whole industry has become on GH, which is ironic, considering that git itself is designed to be decentralized. Good opportunity to think about mirroring your repos somewhere else like Gitea or Gitlab.
- autom4ton 2y agogreat point
- readline_prompt 2y agothese things could happen anywhere though. Gitlab also `rm -rf`ed before remember
- jtriangle 2y agoThe odds of all services rm -rf / at the same time are pretty small to be honest. The point is to have your work in multiple places, such that you're not reliant on a single service.
- nox101 2y agoThey're already mirrored on my hard drive. That's how git works.
- deleted 2y ago[deleted]
- cropcirclbureau 2y agoGithub is more than a remote host for git repositories. It's become one of the major CDNs for software distributions. Github Pages host a majority of static sites that developer use. You won't be able to use Cargo, Nix, Scoop and other package managers right now because their registries have a critical dependency hosted on Github. This is not to mention all the projects that rely on Github for project management, devops, community and support desk. GitHub is also very international, I doubt isolated netziens like those from China are shielded from this outage. I imagine very, very few software shops are unscathed by this. The whole affair is very on brand for 21st century software which is to say pitiful.
- theteleporter 2y agoWtf, thought it was me alone!
- 2YwaZHXV 2y agogiven that it seems like the entire thing is busted, can anyone explain how the unicorn page is being served?
- esrauch 2y agoThey probably have a reverse proxy in front of all their http endpoints and that is still up and able to show the unicorn if the backends aren't responsive. The static content on the error page might also be on akami or cloudflare side.
- 2YwaZHXV 2y agomakes sense, thanks. the images on the page are all just base64 encoded right into the html
- bill_lumbergh 2y agohttps://github.blog/news-insights/the-library/unicorn/ https://github.blog/news-insights/the-library/unicorn/ Unicorn has a slightly different architecture. Instead of the nginx => haproxy => mongrel cluster setup you end up with something like: nginx => shared socket => unicorn worker pools When the Unicorn master starts, it loads our app into memory. As soon as it’s ready to serve requests it forks 16 workers. Those workers then select() on the socket, only serving requests they’re capable of handling. In this way the kernel handles the load balancing for us.
- 2YwaZHXV 2y agoamazing, thanks!
- robertclaus 2y agoSure looks like it!
- dghlsakjg 2y agohttps://www.githubstatus.com/ https://www.githubstatus.com/ This is a pretty good place to check. The lag is pretty minimal traditionally. At the time of posting everything is broken.
- th3w3bmast3r 2y agoThis is my first time seeing the angry unicorn! Hopefully it’ll be gone soon :(
- exfil 2y agoI am happy that my project is pushed also to codeberg.
- lucb1e 2y agoAnd has a website so anyone could just ask me if something went wrong on github's side and I can send them a complete copy. Decentralised version control is nice!
- deleted 2y ago[deleted]
- jezfromfuture 2y ago[dead]
- timetraveller26 2y agoIts all down according to https://www.githubstatus.com/ https://www.githubstatus.com/ Update - Issues is experiencing degraded availability. We are continuing to investigate. Aug 14, 2024 - 23:19 UTC Update - Git Operations is experiencing degraded availability. We are continuing to investigate. Aug 14, 2024 - 23:19 UTC Update - Packages is experiencing degraded availability. We are continuing to investigate. Aug 14, 2024 - 23:18 UTC Update - Copilot is experiencing degraded availability. We are continuing to investigate. Aug 14, 2024 - 23:13 UTC Update - Pages is experiencing degraded availability. We are continuing to investigate. Aug 14, 2024 - 23:12 UTC
- c23gooey 2y agoStatus page showing a complete outage now
- willchen 2y agoI'm wondering why this isn't on the front page? It has a lot of points in 23 minutes.
- mvdtnz 2y agoHN has a strange philosophy built into its ranking algorithm that an item with a large number of comments early on should be de-ranked because the conversation is likely to be of poor quality.
- quincepie 2y agoYet again, this shows how useless GitHub status page is.
- kgrax01 2y agoCould it have been brought down intentionally? Related to this? https://www.bleepingcomputer.com/news/security/github-actions-artifacts-found-leaking-auth-tokens-in-popular-projects/ https://www.bleepingcomputer.com/news/security/github-action...
- bamboozled 2y agoHow would customer credentials being leaked be part of an outage of this size ?
- kgrax01 2y agoIf its enough of a security issue they could have pulled the site while its fixed/cleaned
- bamboozled 2y agoIf that was the case, you'd think it would've been a less dramatic outage right? Maybe even a status page to indicate that?
- fragmede 2y agoBecause there are worse things than being down; if the front page got hacked and is spewing gore or CSAM or PII or creds, for example.
- johnnypangs 2y agoSeems like it was a config change that cause it. They reverted it reality quickly. https://www.githubstatus.com/incidents/kz4khcgdsfdv https://www.githubstatus.com/incidents/kz4khcgdsfdv
- shrimpboyho3 2y ago[flagged]
- BeefySwain 2y agoInteresting that CoPilot is down as well. I would have assumed it was really only part of GitHub as a branding/marketing thing.
- peterlk 2y agoCan't wait for the writeup! So many services down at once... Something very interesting must have happened
- ghostbk24 2y agoon X @githubstatus seems to be getting regular updates / automated messages around impact
- kimboox 2y agoTime to go out and see people
- GDTang 2y agototally down -.- Cannot access from Hong Kong OMG...
- livefish 2y agoImagine everyone having their site on gh pages, and artifacts for OS startup on here. Hello, NixOS...
- GDTang 2y agototally down -.-
- zachdoescode 2y agoI swear even my VSCode intellisense is broken now... Rip to a real one.
- that_other_one 2y agoWelp, that’s as good a time as any to call it a day! Good luck to the devs and dev-analogues involved in getting the ship righted.
- livefish 2y agoImagine everyone having their site hosted on gh pages. Now imagine artifacts for system update on github. Hello NixOS filesystem!
- low_tech_punk 2y agoAll the AI-native developers are twiddling their thumbs because Copilot is out of office.
- croemer 2y agoCause seems to be database related per most recent update (23:29 UTC): > We are experiencing interruptions in multiple public GitHub services. We suspect the impact is due to a database infrastructure related change that we are working on rolling back. Aug 14, 2024 - 23:29 UTC
- dylanz 2y agoWho's the Bozo Doofus maintainer? https://yhbt.net/unicorn/LATEST https://yhbt.net/unicorn/LATEST. I love that we can still see Unicorn in action. I rarely had problems with it back in the day.
- qianli_cs 2y ago"We suspect the impact is due to a database infrastructure related change that we are working on rolling back."
- shashankkoppar 2y agoWe are experiencing interruptions in multiple public GitHub services. We suspect the impact is due to a database infrastructure related change that we are working on rolling back. hope it comes back soon
- neuronexmachina 2y agoLatest update at 23:29 UTC says: "We are experiencing interruptions in multiple public GitHub services. We suspect the impact is due to a database infrastructure related change that we are working on rolling back."
- Gasu16 2y ago[dead]
- deleted 2y ago[deleted]
- shashankkoppar 2y agoWe are experiencing interruptions in multiple public GitHub services. We suspect the impact is due to a database infrastructure related change that we are working on rolling back. hope it is back up soon
- sergiogjr 2y agoYep, angry unicorn. If the copilot debacle wasn't reason enough to make people migrate or diversify the code repo efforts with, let's say, GitLab, this should.
- deleted 2y ago[deleted]
- deleted 2y ago[deleted]
- remram 2y ago@dang https://www.githubstatus.com/incidents/kz4khcgdsfdv https://www.githubstatus.com/incidents/kz4khcgdsfdv is probably a better link for this submission now
- pietroppeter 2y agoI should have looked for this before posting the same comment. Upvoted :)
- unicorner 2y agoI've never seen such a serious outage before. Even GitHub Pages hosted sites aren't accessible.
- robotdragonfire 2y agoit crashed the second i opened this old plugin for blockbench
- jrop 2y agoA coworker and I just had to use `git format-patch` and `git am` to exchange work. Git is super cool!
- remram 2y ago`git bundle` is another option for this (I'm not trying to imply it's preferable)
- robotdragonfire 2y agoit crashed the second i opened a github rep for an old plugin for blockbench
- Duende1 2y agoDown in Vancouver, Canada
- 1attice 2y ago...and we're back
- shcheklein 2y agoSeem to be up again, I also wonder what is was.
- aag01 2y agoback up
- readline_prompt 2y agoand... we're back, at least in Japan region
- inmanturbo 2y agoThey were manually ran a hot patch on the distributed production database and forgot to use a transaction
- wdb 2y agoFeels like Github is down more often than Gitlab
- erksa 2y agoThe mobile app on iOS is a 503 with ``` Received a 503 error. Data returned as a String was: <!DOCTYPE html> <!- - Hello future GitHubber! I bet you're here to remove those nasty inline styles, DRY up these templates and make 'em nice and re-usable, right? Please, don't. https://github.co https://github.co... ``` That's where it's cut off on my screen. Curious what the link is :) I like to think, someone did.
- arjvik 2y agoAnyone seen the full text of the error page?
- goranmoomin 2y agoSeems like sites based on GH Pages were down, but are back up (i.e. the Rust blog).
- inmanturbo 2y agoThey manually ran a patch query on the distributed production database and forgot to use a transaction
- ICHx 2y agoback to live
- zacym 2y agoAug 14, 2024 - 23:29 UTC Update - We are experiencing interruptions in multiple public GitHub services. We suspect the impact is due to a database infrastructure related change that we are working on rolling back.
- frabjoused 2y agoSeems to be back online.
- goranmoomin 2y agoIt feels so wrong that there are so many blogs and websites that are based on GH Pages and they all died at once… Seems like they’re back up though. Or at least the Rust blog is back up.
- omoikane 2y agoMaybe it's fixed already? Works for me.
- wavemode 2y agoI sure wish this had happened before I logged off from work for the day... "Why isn't this project done yet?" "Didn't you hear? GitHub is down!" and I get to go out for a long lunch
- kgrax01 2y agoBack online for me
- mrbluecoat 2y ago[flagged]
- j-wags 2y agoWonder if this is related to the big cyberattack on Iran earlier today https://www.jpost.com/breaking-news/article-814715 https://www.jpost.com/breaking-news/article-814715
- gregors 2y agocurious if their layoffs last year had the intended impact
- Havoc 2y ago[flagged]
- wigwi 2y ago[dead]
- wigwi 2y ago[dead]
- EADDRINUSE 2y agoGH Ops team be like Senior: Ah found it! Let's just rollback one revision on the db. Newguy: let me fix this! `kubectl rollout undo ... --to-revision=1` Newguy: Ok, Started rollback to revision one! Senior: Uh-oh..
- deleted 2y ago[deleted]
- 1vuio0pswjnm7 2y agoGithub mirrors?
- mhh__ 2y agoThis reminds me that for some reason I am logged into my gaming machine's windows store with my GitHub account thanks to the bizarre way that microsoft do auth.
- ChrisArchitect 2y ago00:30 UTC Resolved https://www.githubstatus.com/incidents/kz4khcgdsfdv https://www.githubstatus.com/incidents/kz4khcgdsfdv
- neoyagami 2y agoand I was in the middle of commiting a hot fix D:, I had to push the image directly to the registry D:
- faultywaterfowl 2y agoThey should have used Kardinal: https://github.com/kurtosis-tech/kardinal https://github.com/kurtosis-tech/kardinal
- PoignardAzur 2y agoThis is weird. I've been using Github all night (in France) and didn't notice anything was wrong. Was the outage in North America?
- pietroppeter 2y ago@dang maybe the link could now be updated with this one https://www.githubstatus.com/incidents/kz4khcgdsfdv https://www.githubstatus.com/incidents/kz4khcgdsfdv
- darkangelstorm 2y agoServices that explicitly needed the API were also down, and it wasn't pretty. For example: Minecraft Mod packs that rely on SerializationIsBad all went kerplunk! I'm sure a lot of people were scratching their heads yesterday wondering why they couldn't do anything for a time. What made me laugh though was when the "X is functioning normally" immediately followed by "X is degraded, continuing to monitor" messages that kept popping up then right back to "normal" again, all in the same 30 second timespan... made me giggle
- packetlost 2y agoWonder how much of this is to blame on copilot generated code lol
- t_believ-er873 2y agoUnfortunately, outages happen... This situation is a very good reminder of why having backups and a solid Disaster Recovery plan is crucial. Of course, it’s easy to assume that cloud services are always up, but we should never forget about outages. Setting up automated backups for repos and metadata can save a ton of headaches when things go wrong. Plus, having a Disaster Recovery plan means you’re not stuck waiting for the service to come back online—you can keep working with minimal disruption. Here is a good article on how to prepare for the situations like that, when GitHub is down: https://gitprotect.io/blog/github-restore-and-github-disaster-recovery-scenarios-and-use-cases/ https://gitprotect.io/blog/github-restore-and-github-disaste...
- miloszgp2000 2y agoThis was not the first or last GitHub outage. Unfortunately, this was a big one. That why now it is even more important to have backups of your work and restore capabilities in case of a scenario like this outage. This article sheds light on the importance of backups along with the best practices to follow: https://gitprotect.io/blog/github-backup-best-practices/ https://gitprotect.io/blog/github-backup-best-practices/