6 ms·
I’m not sure at what organization that is true. My company lives out of GitHub and Jira and I’ve hardly noticed the three month surge. GitHub would have to do
by edgyquant 5y ago
I’m not sure at what organization that is true. My company lives out of GitHub and Jira and I’ve hardly noticed the three month surge. GitHub would have to do a lot worse to get many companies to want to host their own services. This is the argument people have said about the cloud from day one.
People want to know it isn’t their problem, that makes cloud computing (and things like GitHub) worth their weight in gold. I have real problems to solve I don’t want to deal with a git repo manager on top of that.
- jeltz 5y agoMaybe you are in a different time zone because our organization certainly noticed and was disrupted by this.
- gjulianm 5y agoAt my organization it's always been true. Setting up GitLab is fairly easy, in my company we do it and it's cheap (on-prem hosting is basically zero, and we had the IPs/domains already) and it hasn't given us too many headaches. I think last time I had to do something was maybe a few months ago when I restarted it so that it picked up the updated SSL certificate.
- xondono 5y agoI’d say it depends, I run my own on prem server and gitlab was a PITA. Too many moving parts, updating took too much of my time, and I never felt “safe”. Moving to gitea solved all of those issues for me (thus far), now I’m looking into adding other stuff like CI through Drone.
- edgyquant 5y agoIt definitely depends. We’re pretty early stage and I’m the senior engineer+infrastructure guy so running our own gitea instance or whatever is just more time that I’m almost out of.
- Inversechi 5y agoDid you consider woodpecker instead of drone? It's basically an evolved fork of the OSS version. https://woodpecker-ci.org/ https://woodpecker-ci.org/
- xondono 5y agoDidn’t even know about it. I’ll check it out. Thanks!
- KronisLV 5y agoCuriously, this was also my own experience! I actually wrote a bit about the migration process, as well as the reasons for migrating over to Gitea, Nexus and Drone CI as opposed to using GitLab, GitLab Registry and GitLab CI: https://blog.kronis.dev/articles/goodbye-gitlab-hello-gitea-nexus-and-drone https://blog.kronis.dev/articles/goodbye-gitlab-hello-gitea-... With containers, it's actually a pretty good experience that's not too hard to setup or manage.
- nightpool 5y agoSelf-hosted GitLab got a good callout yesterday from Microsoft, it appears to be a favorite of LAPSUS$: https://www.microsoft.com/security/blog/2022/03/22/dev-0537-criminal-actor-targeting-organizations-for-data-exfiltration-and-destruction/ https://www.microsoft.com/security/blog/2022/03/22/dev-0537-... Self-hosting always increases the operational burden of making sure your systems are secure. Maybe you have the engineering resources to spend on patching everything immediately and conducting in-house pen tests, but for most companies it's much, much more secure to let the software's developers host it as well.
- temp8964 5y agoNot necessarily. Self-hosted services are protected by company firewall / VPN. They can setup very restrictive network access. They don't have the same level of risks as public services like GitHub or GitLab.
- hhh 5y agoEstablishing an entry point via VPN is Lapsus$ primary first step.
- Melatonic 5y agoExcept that the software developers hosting is also a much, much bigger target and you generally do not have any real control over how often they are patching either.
- skeeter2020 5y ago>> Setting up GitLab is fairly easy, in my company we do it and it's cheap (on-prem hosting is basically zero, and we had the IPs/domains already) In what tech company is hosting or domains the main cost centre? Many companies spend more on a single hour of a dev's time than their entire GH monthly bill.
- capitol_ 5y agoI think we pay about $10 per developer per month for github, and with about 1000 developers I would love that hourly rate.
- kleebeesh 5y ago...What? $10 x 1000 = $10k / month. $10k x 12 = $120k. That is a new grad software engineer salary in any US city. You'd pay more than that for a single dev with the devops and security experience to keep GHE running and patched for 1000 devs.
- tjoff 5y agoWell, considering you'd likely spend an average of 5 minutes per day doing it I wouldn't mind it.
- cortesoft 5y agoThe person was replying to a comment saying they spend more on a SINGLE HOUR of a dev's time than the monthly GH bill, which is not true for an org of more than 20 people or so (depending on hourly rate).
- kleebeesh 5y agoAh, totally misread it. Thanks.
- zeku 5y agoJust a bone to pick... new grad engineers in my US city started around 60-70k in 2018 when my college cohort graduated. Southern US...
- thaeli 5y agoAlso, looking at this it seems like GitHub isn't doing the common SaaS thing of just lying on their status page. Many providers, both internal and external, would look a lot worse if they had honest status pages.
- mirekrusin 5y agoThey are green for good 15 minutes from first moment i see problems, not the first time, it happens actually quite often. Maybe that's the time they need to confirm/cross check/write status update, don't know.
- georgemcbay 5y agoWhile quicker reporting would be better, 15 minutes is anecdotally a lot better than I see from most other services where their status pages will report all-clear hours into full outages.
- thaeli 5y agoYeah, I'm legit impressed with a 15 minute time here.
- judge2020 5y agoThey probably allow regular SREs to trigger an incident on the status page on their own, when the likes of AWS and other bigger cloud providers are rumored to need approval from a VP[0] to update the status page. 0: https://news.ycombinator.com/item?id=29475756 https://news.ycombinator.com/item?id=29475756
- ishanjain28 5y agoThey do intentionally or not lie about this on their status page. From December 25th to December 31st 2021, Github actions had network problems almost every single day for hours and the status page was green out through out that period. Same thing also happened few months back. It feels like they do this manually and it's only done when enough people are effected.
- 5y ago
- jmartens 5y agoMy company monitors the functionality, performance and availability of apps like Github, and we have certainly noticed the increase in issues lately.
- edgyquant 5y agoWe were actually talking about implementing this last week. Not for GitHub but for slack as it seems to have issues once a month or so.
- SkyPuncher 5y ago> I’m not sure at what organization that is true. My company lives out of GitHub and Jira and I’ve hardly noticed the three month surge These have been minor inconveniences for us - at worst. Most of the time it simply means people jump to something else then come back later in the day. Failing tests and PR feedback cycles are more of a blocker to our team than these outages.
- ManWith2Plans 5y agoI will say that for us this is a huge deal. We're a devops services company, and our customers expect their deployment pipelines to work. This is becoming a huge pain-point for a few of our customers and we recommended Github Actions to them. A couple of our customers want us to move away from GitHub actions because of how disruptive outages have been.
- wnevets 5y ago> I’ve hardly noticed the three month surge. This has been my experience as well. I don't know if that means GitHub is being overly transparent about issues or I've just been lucky but I would hate if people punished services for being transparent and informative on their status pages.
- zenexer 5y agoGitHub's outages have hit me hard over the past week or so. I don't think it's a matter of them being transparent--if anything, I was hitting errors well before their status page updated. Yesterday it was completely unusable for much of my workday, and today tasks that normally take me a few minutes have been taking hours.
- jhugo 5y agoI think the impact was for some reason not consistent between users (maybe due to geographical factors or maybe sharding of accounts?). We're in Asia and I think we've had three different days recently where we couldn't actually get much work done due to GitHub being flakey or down for the entire day and our CI/CD and development processes being built around it. We ended up moving off GitHub onto a self-hosted system, which took about a day of work for one engineer (CI/CD itself was already self-hosted, so just Git, issues and PRs), and there have already been two more GitHub outages since then.
- jacobr 5y ago20 PRs waiting in line for half a day to be merged is pretty annoying. We’ve had that on multiple occasions the last few weeks due to GitHub incidents.