6 ms·
GitHub Down with a 503
- ereckers 12y agoI was waiting for this post :) Let's see who the Saturday team is!
- StepR 12y agoOh damn. The Github team is in for a rude awakening...
- simplon 12y agoYeah seems so.. https://status.github.com/ https://status.github.com/
- StepR 12y agoThats what the link above goes to
- dmoy 12y agoRandom somewhat on-topic question. How does one measure ones technical capability to keep up services that are more available and dependent than third party solutions? I mean obviously starting out, almost everybody can benefit from doing a lot of stuff not in-house, but at some point you'd be able to do better yourselves. But then maybe for certain services from certain providers, raw technical capability never catches up? How do you know when that point is? Is there even data out there on how frequently big providers go down? (Not just Github, but stuff like aws, etc)
- bricestacey 12y agoBy the time you're talking about 99.9% uptime there is only 9 hours downtime a year so there isn't much wiggle room for failure. There is never a reason any one person should assume they can do better than a third party because you're likely sleeping 2920 hours a year. Basically, you're never good enough on your own. If you care about uptime you're going to need to pay good money for it.
- increment_i 12y agoBut, hypothetically speaking as of now, could this be automated? And if so, how?
- __david__ 12y agoRedundancy. Lots of it. A simple example would be hitting your website with a script every so often and when it stops responding then update your DNS to point to another data center that's up and ready to go.
- bricestacey 12y agoIn short, no, unless you think people replacing broken hardware or patching bugs in your software can be automated. Your question is too vague. Github is up enough that I don't care. However, it's down enough I wouldn't want to not be able to deploy because it's down. Therefore, I may mirror my repo somewhere else. That's easy because git is decentralized. It's a lot cheaper than running some alternative that I guarantee is always running. You can do this by simply pushing to a mirrored remote branch. If you're just interested in the subject, research high availability.
- bricestacey 12y agoIn short, no, unless you think people replacing broken hardware or patching bugs in your software can be automated. Your question is too vague. Github is up enough that I don't care. However, it's down enough I wouldn't want to not be able to deploy because it's down. Therefore, I may mirror my repo somewhere else. That's easy because git is decentralized. It's a lot cheaper than running some alternative that I guarantee is always running. You can do this by simply pushing to a mirrored remote branch. If you're just interested in the subject, research high availability.
- joshdotsmith 12y agoRight in the middle of a major push to production. Awesome.
- akerl_ 12y agoIt's too bad git requires a central server.
- AYBABTME 12y agoHow is this a useful comment? Everybody knows that Git is distributed. Everytime Github's down, somebody feels the urge to remind everybody else about that. Github's not only Git. Lots of services integrate with Github. Stuff like continuous integration, continuous deployment. If your system is built on those things, Github being down will prevent you from deploying. What's the alternative, replicate everything in-house/self hosted? Should startup stop using third party service providers?
- akerl_ 12y agoIf your workflow involves services that integrate with GitHub, you should have a plan for when GitHub is not available. You've got your source code elsewhere, you should have docs and procedures for running the tests in your own dev environments and for utilizing whatever staging/deploy process you prefer without GitHub as an intermediary. I continue to post comments like that because while I love GitHub, watching the community put all their eggs in GitHub's basket, especially when GitHub lives on top of a tool designed to avoid SPoFs, is concerning.
- hamburglar 12y ago100%. GitHub is amazing. GitHub is a wonderful tool. GitHub is also just a service cooked up by a handful of jokers who got enough funding to make it happen, and they do a reasonable job of keeping it running most of the time. I wish them all the best, and yet making my own company's infrastructure reliant on it is not something I'm comfortable with. Sometimes they're going to blow it, and that's ok. If them blowing it means my business is affected, I'm the dummy.
- aroman 12y agoI was wondering why my `bower install` was failing... :/
- qntmfred 12y agoDoesn't bower install from a local cache if network request fails? npm on the other hand... per docs at https://github.com/bower/bower https://github.com/bower/bower oh....
- aroman 12y agoYeah it does pull from cache, but I didn't have the packaged cached yet. As for the docs, it is somewhat crazy to think how much documentation is hosted by GitHub these days...
- cclogg 12y agoHaha, I wonder how many of us came here because we couldn't push or pull commits, so just taking a HN reading break... and boom the top post is about Github!
- mafro 12y agoWhen Github appears down, I come to HN to find out if it's affecting everyone :p
- YoukaiCountry 12y ago'Gee, github is down. Is it just me? Better check HN!' Exactly what I just did too.
- pwelch 12y agoThis is what I do as we'll. :)
- bdcravens 12y agoI first check their Twitter account, and then when there's nothing there about an outage (currently isn't), I come here.
- deleted 12y ago[deleted]
- skrebbel 12y ago
- wildmXranat 12y agoYep. All metrics down and exception rate is up gentlemen: http://imgur.com/Ncm5Rrx http://imgur.com/Ncm5Rrx . I just noticed when SQLMap install was hung and every tarball/zip download on their site also leads to Github , oy vey!
- AYBABTME 12y agoAt least webhooks are becoming faster..!
- deleted 12y ago[deleted]
- nxbtch 12y agoyou are really the early bird.
- cheese1756 12y agoI was setting up a new repo, and then everything stopped working. Lucky timing, I guess.
- OutThisLife 12y agoIt's all over now.
- mhahn 12y agoall the overachieving engineers working on a saturday...
- danielhellier 12y agoCurious, which chart lib is GitHub using for those charts?
- cheese1756 12y agoThey make calls to d3 in the chart snippet, so I assume it's that. Here's the relevant code from application.js: http://pastebin.com/ApL9SxXs http://pastebin.com/ApL9SxXs
- misaelm 12y agoI was curious too and noticed that those are SVG charts so after a quick check on the minified application.js it seems like it's D3.
- argonaut 12y agoReminds me of the classic bumbling quote: "Why don't they just host GitHub where they host the status page?"
- cshimmin 12y ago"Why don't they make the whole damn plane out of the black box?"
- altcognito 12y agoI'm always impressed that life support systems on star trek fail, but artificial-gravity never ceases to work.
- lloeki 12y agoProbably there are a handful of gravitons or something embedded right in the floor, or there is massive inertia to the system. We tend to forget that not all solutions require active power.
- larrybolt 12y agoThis does makes one wonder that if you need to make an app that has as little downtime as possible perhaps the best solution is to port as much as possible functionality client-side with client-side caching of data. (theoretically, if I could cache every repo I visit using squid-proxy, even if github has an hour downtime I can still access the repo's I visited in the past, not perfect, but it's something!)
- mattkrea 12y agoWhy have they gone down so often in the past few months? Is this another DDOS? I've personally wondered about their stack but I imagine that has little to do with that--mostly Ruby though right?
- sinaa 12y agoIt seems to be MySQL related this time: "github.com is currently unreachable. We are investigating mysql cluster issues."
- papercruncher 12y agoTempted to deploy via rsync on my laptop
- iancarroll 12y agoand right when I wanted to clone something. damn.
- co2 12y agoI'm wondering if this will give bitbucket a boost in unique visits.
- martinvol 12y agoException percentage just got over 100%, new record! (?) http://i57.tinypic.com/4hphtd.png http://i57.tinypic.com/4hphtd.png
- saurik 12y agoIt is now up to 175% a couple minutes later (and 205% a couple minutes after that). The issue seems to be that it is actually currently more like 10000%, and is being averaged down throughout the month to a mere 100-200%. What is this even measuring?!
- helper 12y agoIf I were implementing something tracking the rate of exceptions a reasonable thing to measure it against is the total number of requests. In such a system, if each request generated multiple exceptions you would have >100% exception rate.
- viclou 12y agohttps://twitter.com/shit_hn_says/status/421116237792284672 https://twitter.com/shit_hn_says/status/421116237792284672
- robbles 12y agoCould Github store mirrors of all repositories on some third-party service, like S3? Even if they were only updated every 24 hours and had a limited history for each repo, it still seems like it would be a really useful fallback that they could put up when things like this happen.
- eli 12y agoIt's actually not that hard to set up your git to push to both Github and another repository at the same time.
- robbles 12y agoSure, it's easy to do with your own code. But what about apps that don't vendor their dependencies? If you need to pull in third-party dependencies when you deploy or test, it's still possible to maintain your own mirror, but it would be nice if you could depend on the code being "up".
- 4dl0v3-p34c3 12y agoI hope it was just an error or something. Nothhing political, like this: https://gist.github.com/kyledrake/e6046644115f185f7af0 https://gist.github.com/kyledrake/e6046644115f185f7af0 More info: http://www.theverge.com/policy/2014/5/9/5699510/web-hosting-company-puts-fcc-in-slow-lane http://www.theverge.com/policy/2014/5/9/5699510/web-hosting-...
- hayksaakian 12y agothe gist link is also a 503
- Aqueous 12y agoAlright...well, it looks like this whole programming thing was a bust. Was good while it lasted, but yeah, uh, I guess I'm gonna go home, er, something - so...see you guys later?
- mrmondo 12y agoIt's amazing how often github actually goes down - 90% of the time they don't log it on their uptime / status page - it got so bad at one point I added it to our Nagios monitoring to prove a point to our devs that they shouldn't rely on external services for builds when our internal git(lab) server hasn't gone down once this year.
- copperx 12y agoHow does one depend on Github for builds if git is, um, distributed?
- nodesocket 12y ago> We are investigating mysql cluster issues.
- zcs 12y agoIt's back!
- ChiChou 12y agoSomehow I opened the GitHub Status Page subconsciously early this morning, then everything suddenly became 503 in less than 10 minutes! I still have the tab opened now, saying "All systems operational"...
- ChiChou 12y agoSeems like it's back now. (3:13 UTC)
- deleted 12y ago[deleted]
- deleted 12y ago[deleted]
- thamaraimedia 12y agoGitHub now available.
- deleted 12y ago[deleted]