4 ms·
14 Days ago, they went down [0]. And today it's happening again. Twice in less than a month. Another reminder to self host via solutions like GitLab or Gitea.
by rvz 6y ago
14 Days ago, they went down [0]. And today it's happening again. Twice in less than a month.
Another reminder to self host via solutions like GitLab or Gitea. [1]
[0] https://news.ycombinator.com/item?id=23675864 https://news.ycombinator.com/item?id=23675864
[1] https://news.ycombinator.com/item?id=23676072 https://news.ycombinator.com/item?id=23676072
- dtech 6y agoself-hosting requires quite a bit of scale to be more reliable, otherwise you'll most likely still have possibly longer outages, just at different times.
- flukus 6y agoVery often that's ok if you can choose when the outages occur. With most you can because outages tend to be for regular maintenance and upgrades far more often than hardware failures. A small geographically concentrated company only needs 9 fives. Even for bigger companies that need more uptime they can at least give a shout out to everyone that it will be unavailable from X to Y, it doesn't just go down randomly with no notice whenever github decides to push a release.
- gregoriol 6y agoIf you think self-hosting doesn't randomly go down, you have never done self-hosting. In the last few months we had a server that physically crashed and had to be replaced, we had a bad kernel that paniced randomly and a gitlab upgrade that failed in the middle. Is you team equiped and ready to handle these? If not, don't self-host anything critical to your company, or your costs, downtimes and losses will be much much higher.
- dijit 6y agothis always comes up, and it's one of those things that's obviously true on the surface until you understand context. I have a self-hosted gitlab instance and it has much higher uptime than github in the last year. At some point a weekly downtime is going to trump any potential random outages. The major difference might be your ability to affect those outages (positive or negative) because humans are the primary cause of outages, and you can restore service to a gimped passable state if its your own thing.
- vasachi 6y agoI mean, hardware nowadays is quite reliable. We have a simple server with three disks in raid1, and it basically just works. Gitlab has regular backups, so at worst we would have one day of lost work.
- Symbiote 6y agoThat surely depends how much they use Github, and when they need the service available. Our self-hosted build and artifact servers have not gone down without planning in many years. Planned outages are brief, for a small team it's not inconvenient to announce a 10 minute outage for an upgrade.
- rplnt 6y agoYou can self-host GitHub too. Or BitBucket.
- jaekash 6y ago> Or BitBucket But why would anybody ever do that?
- moooo99 6y agoI actually enjoyed BitBucket for smaller projects. They offered free private-repos earlier than GitHub did, so that was the main sellingpoint for me. The self hosted version of BitBucket comes in at a pretty reasonable price. I don't know that for sure (and I can't check thanks to GH being down) but I'd imagine the self-hosted versions for GitHub Enterprise are quite a bit more expensive than Bitbucket or Gitlab.
- rplnt 6y agoIt's surprisingly easy to set up and run, even with redundancy, integrates well with Jira, Confluence. Was fast enough, stable enough, had everything one needed (offered more than a hosted solution such as github). IIRC kept git completely separate, so it worked even during updates or restarts. Disclaimer: I passionately hate Atlassian products, but I don't feel like using Stash/Bitbucket is a bad choice in any way.
- gregoriol 6y agoCould you provide details on how you plan to be more reliable with a self-hosted solution? what kind of archtecture would you use? how many people will be involved in maintaining?
- creshal 6y agoGit is decentralized by default, so all you really need is for your self-hosted version to not crash at the same time as github does. That's not too hard to pull off.
- gregoriol 6y agoIt's not that I'm against, the contrary, but in the last months I've experienced a physical server crash, a random kernel panic and a failed upgrade of GitLab: that all took 10/20 hours of work, didn't result in any data loss, but the downtime was clearly higher than what GitHub has (+ nobody on the team was on holidays at that moment), plus any maintenance like software or system upgrades has to be done outside team working hours to limit the impact.. real self-hosting is not that easy!
- ashildr 6y agoA failed upgrade of GitLab? Can you give details - I got used to monthly Gitlab updates going very smoothly and now I’m a little afraid. What went wrong?
- gregoriol 6y agoLast one we had was because the ssh connection was lost during the apt upgrade (gitlab omnibus ce). Another one I remember was when some new configuration key was needed and prevented one of the components from starting (took ~2 hours to find what it was). It doesn't happen often, don't worry that much though, gitlab omnibus works well, but is hard to investigate/fix because it's huge and not a tech stack we do understand well since we mostly use gitlab-ctl commands, so when something goes wrong it's hard
- aabbcc1241 6y ago
- dt3ft 6y agoI would choose self-hosting for small to medium size teams any day. I can't fathom why people choose not to self-host at this scale. Your data. Your control. Your network. Your infrastructure. Your responsibility. Are people becoming more afraid of responsibility these days?
- icemelt8 6y agomoney.
- rorykoehler 6y agoIt won't help bring in revenue. Pointless to worry about that at the start
- dtech 6y agoSmall to medium size teams should be hyper-focused on developing their product and bringing in customers and revenue. Spending time on commoditized infrastructure is only a good idea if that saves time in the short to medium term. But hey, with the number of outages GitHub has recently that break even point is coming closer and closer.
- jaekash 6y agoSelf hosting quickly becomes a mess of weird gatekeeping and bad maintenance. Almost never have I seed well maintained self hosted infrastructure. Most of the time it is installed, gatekept and then left to bitrot without any upgrades or maintenance.
- douglasheriot 6y ago> Your control. Your network. Your infrastructure. Your responsibility These are all the reasons why I don't want to self-host my own git. I can live with it being down every now and then. And when it is down, I don't want it to be my job to fix it. I've got more important stuff to worry about.
- sergiotapia 6y agoit's literally a money sink for wishy-washy benefits if you're a small/medium sized team. It's why heroku became a billion dollar company, time+money is better spent elsewhere at the start.
- trulyrandom 6y agoSelf hosting GitHub is an option as well. We've been self hosting GitHub Enterprise for years and it has had no downtime other than during scheduled updates (at night, when nobody's at work).
- jaekash 6y ago> We've been self hosting GitHub Enterprise for years and it has had no downtime other than during scheduled updates (at night, when nobody's at work). Is it free to self host like GitLab or Gitea? If not, what does it offer that you can't get from those?
- joelhaasnoot 6y agoOn-premise or own-cloud hosting is quite import for some customers.
- jaekash 6y ago> On-premise or own-cloud hosting is quite import for some customers. Yes it is, and GitLab and Gitea both offer that, which brings us back to the question I asked, what does GitHub Enterprise offer that you can't get from GitLab and Gitea?
- reificator 6y ago> > Is it free to self host like GitLab or Gitea? If not, what does it offer that you can't get from those? > On-premise or own-cloud hosting is quite import for some customers. Yes... it is quite important- That's why people use things like Gitlab or Gitea.... But back to the original question, what does Github Enterprise offer that you can't get from Gitlab or Gitea?
- justincormack 6y agoNo it is not free, it is GitHub's main source of revenue.