6 ms·

"just run your own git server" comments always make me scratch my head. IMO folks managing services for personal use vastly underestimate how much harder every

by pid-1 5y ago

"just run your own git server" comments always make me scratch my head. IMO folks managing services for personal use vastly underestimate how much harder everything can get in an actual business environment.

nijave 5y ago
I don't think it's necessarily just "personal use". The complexity quickly scales with the company size. Replacing Github with Gitlab might be a week-long project at a small company but could easily turn into a multi-year project at a large one. It's not just a technical problem, either. Bigger companies tend to have more expertise-oriented teams (security, compliance, developer tooling, operations, internal infrastructure) which tends to make decisions more difficult than when a single person or team can do it themselves.
- vlunkr 5y ago
  > Replacing Github with Gitlab might be a week-long project at a small company A week-long project initially. Now you have to install updates, set up and maintain secure access, reboot or troubleshoot when it dies, etc. Installing things is the easy part.
  - netizen-936824 5y ago
    What kind of a company doesn't have at least one sysadmin, I feel like that's kind of a critical position for maintaining systems?
    - buffet_overflow 5y ago
      The problem is when they have more services than sysadmins. While the sysadmins are busy upgrading the git server, the logging infrastructure suffers. They pivot to work on that, now the CI/CD server is down/slow/randomly breaking. But the sysadmin that knew the ins and outs of it left last quarter so the new sysadmins don't want to touch it. Oh, and management doesn't prioritize any of this stuff, so actually jumping two versions on the git server is a much bigger, fragile ordeal now than it was a month ago.
      - CameronNemo 5y ago
        Exactly. Our team of 6 sysadmins manages: - DNS appliances, storage appliances, NTP appliances, - hypervisors, Dev/stage/prod k8s clusters, some other k8s clusters - dev/prod Elasticsearch/Logstash/Kibana clusters - internal GitLab, Jira, Confluence, nautobot, OpenDCIM, a deprecated Twiki - several internal custom apps - Probably more I am forgetting. Nothing gets patched consistently. Everything is neglected to a certain degree.
        rhizome 5y ago
        I'm not going to die on this hill, but that seems like a lot of complexity for a company with in-house skills, maybe even the worst of both (in-house vs cloud/managed) worlds.
        stevekemp 5y ago
        In a previous job I managed a migration from bitbucket (!) to self-hosted Github Enterprize. As part of the migration I forced the creation of a monthly "backup restore test". Every month we had a recipe to spin up a new instance of the Github appliance, and import the most recent backup into it. A lot of systems do tend to get a little neglected over time, and of course different organizations have different priorities, but I insisted on this because I figured for that particular company the Github instance was one of the most critical components - if it is down people can't work.
    - icedchai 5y ago
      You'd be surprised. Most dedicated sysadmins and DBA positions were done away with as part of the "devops" movement, especially as everything moved to the cloud and there were no longer physical systems to maintain. Developers are smart, they can just do that work too, right? It's all just typing. /s
    - coward123 5y ago
      Loads of non-profits and small businesses that have surprisingly demanding tech needs but can't afford / don't understand / can't manage / have only a part-time need, etc.
      - netizen-936824 5y ago
        Its literally essential to have solid tech these days. I don't understand how businesses think they can operate while skimping or even skipping a halfway decently funded IT dept
        WJW 5y ago
        It's not even close to essential, since by far the majority of businesses are running without anything close to what people on HN would consider adequate. It only becomes a problem if too many of your competitors have good tech and even then only if your industry depends on (software) tech. Companies like bakeries and building contractors can run fine with shitty IT and even if their competitors do IT better, no customer is going to drive further to get a bread from someone who has proper backups instead of from the closest shop.
        throw0101a 5y ago
        > I don't understand how businesses think they can operate while skimping or even skipping a halfway decently funded IT dept If a business does not have someone knowledge about IT, how can they know that they have 'deficiencies' with regards to IT? Things are the way they are and they think that's 'normal' because they don't know any better.
        nijave 5y ago
        Big business: We can look at our SIEM and endpoint management dashboards and determine that our end user machines are reaching end of useful life due to resource consumption and failure frequency Small business: Please IT help my 5 year old laptop can't keep up and I swear it's not a virus. Yes, I updated everything It's amazing watching friends/family outside of tech work. Within 2 minutes of watching someone's machine slow to a crawl, I open Task Manager on Windows and see they're out of RAM and their machine is heavily paging: Yeah, if you had another $40 8GB stick of RAM your machine would be significantly more responsive.
    - throw0101a 5y ago
      > What kind of a company doesn't have at least one sysadmin, I feel like that's kind of a critical position for maintaining systems? Even if they have one, that sysadmin could be incompetent or out of their depth. Just started a new job where the last guy was decent enough, but what ended up being built up needs to basically be redone, e.g., a giant /21 network: no server VLAN/subnet, or separate network for network management interfaces or server IPMI.
      - vlunkr 5y ago
        > could be incompetent or out of their depth Or just overworked. In theory you can self-host everything, but there's a always a time tradeoff.
    - nijave 5y ago
      Haven't worked at a startup yet with a sysadmin. They either have devops or SREs take care of infrastructure but those types tend to be cross functional so they're also handling infrastructure, operations, incident response, developer tooling, CI/CD environments, code performance issues (basically everything that's not application development). One company I was at heavily used SaaS and AWS for everything so there weren't very many systems to manage. We'd just jam everything in Docker containers and use AWS ECS. Patching hosts was just rotating to a new AMI on AWS; patching applications was updating dependencies.
  - coward123 5y ago
    No kidding. I would have preferred to migrate the client to GitHub, but they were convinced that wasn't acceptable - had to run their own GitLab. Turned into a mess of managing Gitlab updates and BS rather than working on the product.
marginalia_nu 5y ago
A git server is just a computer with ssh access, though. Git itself is designed in a way where it doesn't even need a server. If you keep it simple, it stays simple.
- nijave 5y ago
  Sure but the article isn't talking about a "git" server. It's talking about replacing Github with Gitlab which is a fully integrated developer workflow tool with code review functionality. They use "Git server" a lot in the article, but the opening and closing both specifically mention a Github replacement.
  - marginalia_nu 5y ago
    Right, but git itself has no concept of code reviews, if that's what you want, maybe you shouldn't be looking for a git server, but some mechanism for code reviews.
    - jeltz 5y ago
      Yes? That is presumably exactly what they are looking for. They just worded it poorly. The reason people use GitHub and Gitlab is usually not because they want a git server. For that there are much better tools like gitolite.
- maccard 5y ago
  A postgres server is just a computer with 5432 open instead of 22. > If you keep it simple, it stays simple. Things that work well for one person in isolation don't work at scale or for teams. How do you handle authentication for your git server? What about backups? Manage disk space? Updates? That's before you get to the point of dealing with workflows and integrations, or "it's slow when 10 people clone at the same time"
  - marginalia_nu 5y ago
    Every cloned git repository is a backup. That's like the entire point of git. It's decentralized version control. This is why the notion of a "git server" is a bit of an oxymoron. Most of the things you're struggling to solve are effectively preventing it from actually working as intended.
deleted 5y ago
[deleted]

by pid-1 5y ago

nijave 5y ago

I don't think it's necessarily just "personal use". The complexity quickly scales with the company size. Replacing Github with Gitlab might be a week-long project at a small company but could easily turn into a multi-year project at a large one. It's not just a technical problem, either. Bigger companies tend to have more expertise-oriented teams (security, compliance, developer tooling, operations, internal infrastructure) which tends to make decisions more difficult than when a single person or team can do it themselves.

vlunkr 5y ago
> Replacing Github with Gitlab might be a week-long project at a small company A week-long project initially. Now you have to install updates, set up and maintain secure access, reboot or troubleshoot when it dies, etc. Installing things is the easy part.
- netizen-936824 5y ago
  What kind of a company doesn't have at least one sysadmin, I feel like that's kind of a critical position for maintaining systems?
  - buffet_overflow 5y ago
    The problem is when they have more services than sysadmins. While the sysadmins are busy upgrading the git server, the logging infrastructure suffers. They pivot to work on that, now the CI/CD server is down/slow/randomly breaking. But the sysadmin that knew the ins and outs of it left last quarter so the new sysadmins don't want to touch it. Oh, and management doesn't prioritize any of this stuff, so actually jumping two versions on the git server is a much bigger, fragile ordeal now than it was a month ago.
    - CameronNemo 5y ago
      Exactly. Our team of 6 sysadmins manages: - DNS appliances, storage appliances, NTP appliances, - hypervisors, Dev/stage/prod k8s clusters, some other k8s clusters - dev/prod Elasticsearch/Logstash/Kibana clusters - internal GitLab, Jira, Confluence, nautobot, OpenDCIM, a deprecated Twiki - several internal custom apps - Probably more I am forgetting. Nothing gets patched consistently. Everything is neglected to a certain degree.
      - rhizome 5y ago
        I'm not going to die on this hill, but that seems like a lot of complexity for a company with in-house skills, maybe even the worst of both (in-house vs cloud/managed) worlds.
      - stevekemp 5y ago
        In a previous job I managed a migration from bitbucket (!) to self-hosted Github Enterprize. As part of the migration I forced the creation of a monthly "backup restore test". Every month we had a recipe to spin up a new instance of the Github appliance, and import the most recent backup into it. A lot of systems do tend to get a little neglected over time, and of course different organizations have different priorities, but I insisted on this because I figured for that particular company the Github instance was one of the most critical components - if it is down people can't work.
  - icedchai 5y ago
    You'd be surprised. Most dedicated sysadmins and DBA positions were done away with as part of the "devops" movement, especially as everything moved to the cloud and there were no longer physical systems to maintain. Developers are smart, they can just do that work too, right? It's all just typing. /s
  - coward123 5y ago
    Loads of non-profits and small businesses that have surprisingly demanding tech needs but can't afford / don't understand / can't manage / have only a part-time need, etc.
    - netizen-936824 5y ago
      Its literally essential to have solid tech these days. I don't understand how businesses think they can operate while skimping or even skipping a halfway decently funded IT dept
      - WJW 5y ago
        It's not even close to essential, since by far the majority of businesses are running without anything close to what people on HN would consider adequate. It only becomes a problem if too many of your competitors have good tech and even then only if your industry depends on (software) tech. Companies like bakeries and building contractors can run fine with shitty IT and even if their competitors do IT better, no customer is going to drive further to get a bread from someone who has proper backups instead of from the closest shop.
      - throw0101a 5y ago
        > I don't understand how businesses think they can operate while skimping or even skipping a halfway decently funded IT dept If a business does not have someone knowledge about IT, how can they know that they have 'deficiencies' with regards to IT? Things are the way they are and they think that's 'normal' because they don't know any better.
        nijave 5y ago
        Big business: We can look at our SIEM and endpoint management dashboards and determine that our end user machines are reaching end of useful life due to resource consumption and failure frequency Small business: Please IT help my 5 year old laptop can't keep up and I swear it's not a virus. Yes, I updated everything It's amazing watching friends/family outside of tech work. Within 2 minutes of watching someone's machine slow to a crawl, I open Task Manager on Windows and see they're out of RAM and their machine is heavily paging: Yeah, if you had another $40 8GB stick of RAM your machine would be significantly more responsive.
  - throw0101a 5y ago
    > What kind of a company doesn't have at least one sysadmin, I feel like that's kind of a critical position for maintaining systems? Even if they have one, that sysadmin could be incompetent or out of their depth. Just started a new job where the last guy was decent enough, but what ended up being built up needs to basically be redone, e.g., a giant /21 network: no server VLAN/subnet, or separate network for network management interfaces or server IPMI.
    - vlunkr 5y ago
      > could be incompetent or out of their depth Or just overworked. In theory you can self-host everything, but there's a always a time tradeoff.
  - nijave 5y ago
    Haven't worked at a startup yet with a sysadmin. They either have devops or SREs take care of infrastructure but those types tend to be cross functional so they're also handling infrastructure, operations, incident response, developer tooling, CI/CD environments, code performance issues (basically everything that's not application development). One company I was at heavily used SaaS and AWS for everything so there weren't very many systems to manage. We'd just jam everything in Docker containers and use AWS ECS. Patching hosts was just rotating to a new AMI on AWS; patching applications was updating dependencies.
- coward123 5y ago
  No kidding. I would have preferred to migrate the client to GitHub, but they were convinced that wasn't acceptable - had to run their own GitLab. Turned into a mess of managing Gitlab updates and BS rather than working on the product.

marginalia_nu 5y ago

A git server is just a computer with ssh access, though. Git itself is designed in a way where it doesn't even need a server. If you keep it simple, it stays simple.

nijave 5y ago
Sure but the article isn't talking about a "git" server. It's talking about replacing Github with Gitlab which is a fully integrated developer workflow tool with code review functionality. They use "Git server" a lot in the article, but the opening and closing both specifically mention a Github replacement.
- marginalia_nu 5y ago
  Right, but git itself has no concept of code reviews, if that's what you want, maybe you shouldn't be looking for a git server, but some mechanism for code reviews.
  - jeltz 5y ago
    Yes? That is presumably exactly what they are looking for. They just worded it poorly. The reason people use GitHub and Gitlab is usually not because they want a git server. For that there are much better tools like gitolite.
maccard 5y ago
A postgres server is just a computer with 5432 open instead of 22. > If you keep it simple, it stays simple. Things that work well for one person in isolation don't work at scale or for teams. How do you handle authentication for your git server? What about backups? Manage disk space? Updates? That's before you get to the point of dealing with workflows and integrations, or "it's slow when 10 people clone at the same time"
- marginalia_nu 5y ago
  Every cloned git repository is a backup. That's like the entire point of git. It's decentralized version control. This is why the notion of a "git server" is a bit of an oxymoron. Most of the things you're struggling to solve are effectively preventing it from actually working as intended.

deleted 5y ago

[deleted]