5 ms·

Making GitLab faster

215 points by sytse 11y ago

sytse 11y ago
Previous discussion earlier today in https://news.ycombinator.com/item?id=11173254 https://news.ycombinator.com/item?id=11173254
deleted 11y ago
[deleted]
- sytse 11y ago
  Yes, it is much faster, see https://about.gitlab.com/2016/02/22/gitlab-8-5-released/ https://about.gitlab.com/2016/02/22/gitlab-8-5-released/ and https://gitlab.com/gitlab-com/operations/42 https://gitlab.com/gitlab-com/operations/42
sergiotapia 11y ago
Where can I find this tool Sherlock you guys mention? It sounds interesting.
- YorickPeterse 11y ago
  It's built into GitLab itself, it's not an external tool you can install without using GitLab. Some information about it can be found at http://doc.gitlab.com/ce/development/profiling.html#sherlock http://doc.gitlab.com/ce/development/profiling.html#sherlock.
dogweather 11y ago
Awesome - really happy to see improvements here. It looks like the front-end can use some attention too: PageSpeed Score is only C (79%): https://gtmetrix.com/reports/gitlab.com/57l7nHlc https://gtmetrix.com/reports/gitlab.com/57l7nHlc This is the most valuable measurement, IMO. Server response time is just a proxy for end-user experience.
- sytse 11y ago
  Thanks for caring, we'll have a look at this.
- jakecodes 11y ago
  Thanks for taking a look at our frontend speed. I've created a few issues so we can take a look at this. https://gitlab.com/gitlab-org/gitlab-ce/issues/13820 https://gitlab.com/gitlab-org/gitlab-ce/issues/13820
neom 11y ago
Is the GitLab CEO here?
- gkoberger 11y ago
  CEO is the poster, 'sytse'.
  - sytse 11y ago
    lol
- sciurus 11y ago
  In case you're like me and didn't get the joke at first: https://www.google.com/search?q=%22gitlab+ceo+here%22&ie=utf-8&oe=utf-8 https://www.google.com/search?q=%22gitlab+ceo+here%22&ie=utf...
OSButler 11y ago
I was running GitLab on a VPS with specs close to the lower end of the recommended requirements and its performance would gradually slow down over the course of the day until pages seemed to never finish loading. Restarting gitlab got it responding properly again, but the whole process would repeat over and over again, so that it had to be restarted every couple of hours just to be able to use it. I then switched providers, which also came with more system resources, and GitLab was working without any issues right away. It could have been caused by an oversold node at the previous host, or the slightly higher system specs at the new host made the difference, but I've been running GitLab on that new server for a couple of months now without experiencing those slowdowns again. If you've noticed a gradual slowdown to the point of pages getting stuck on loading indefinitely, then try to up the system specs a bit, as it could make quite the difference.
- sytse 11y ago
  What can also help is running the Omnibus package that restarts processes that start using too much memory automatically.
  - parfe 11y ago
    The Ruby on rails solution to memory leaks probably shouldn't be the goto remedy.
    - rgbrenner 11y ago
      gitlab IS a ruby on rails app Why wouldn't a ruby on rails solution be right for a ruby on rails app?
      - prawnsalad 11y ago
        If the ruby on rails solution to memory leaks is to restart, then it's far better to actually get it fixed to not leak memory in the first place. Restarting is just ignoring the issue.
    - sytse 11y ago
victor9000 11y ago
I finally decided to give GitLab a shot, and so far things are going pretty well. You can make an account with your GitHub login, and you can migrate projects with just one click. From there I just had to set a password for my account, and now I'm pushing commits with no trouble at all. The whole process took just a few minutes, pretty cool!
- sytse 11y ago
  Glad to hear that!
9erdelta 11y ago
This is great! I moved my team's repos to GitLab last year and I love it. Pricing is better, there's more granularity in assigning team member roles, and the mobile app is good too.
- sytse 11y ago
  Great to hear that Matt!
shaunol 11y ago
Kudos to the GitLab team. I've been running it for 12 months now on premises. The Omnibus package makes the instance unbelievable easy to install and upgrade, so there's no excuse not to run the latest versions. I've just been watching my instance get better and better over those 12 months, better UI, more features and better performance. I really love the software and can't talk it up enough.
- sytse 11y ago
  Thanks shaunol, glad to hear that!
shadeslayer 11y ago
I've been using gitlab for the past few weeks to host a couple of repositories, it's awesome so far since it's the only solution that allowed me to mirror a repo AND create my own branch on the mirrored repository. The only complaint I really have is the fact that I have to append a ".git" to the http url to clone the repository. Alternatively, it'd be nice to have a git:// read only url too.
- sytse 11y ago
  Feel free to open an issue both for the http clone without .git, it seems interesting but I'm not sure how complicated it is. Not sure about the git:// read only url, I don't oversee the consequences.
  - mintplant 11y ago
    git has a default user agent that looks like git/2.0.0, so you could detect that and redirect. I believe this is how GitHub makes this work.
  - jacobvosmaer 11y ago
    The git:// protocol requires its own daemon (which is bundled with Git) running on port 9418 https://git-scm.com/book/en/v2/Git-on-the-Server-The-Protocols#The-Git-Protocol https://git-scm.com/book/en/v2/Git-on-the-Server-The-Protoco... . The standard Git daemon is unsuitable for GitLab I think because of the way it manages access control. https://git-scm.com/book/en/v2/Git-on-the-Server-Git-Daemon https://git-scm.com/book/en/v2/Git-on-the-Server-Git-Daemon For this to work in GitLab we would need to find or write an alternative implementation of the protocol. Sounds like a lot of work to me when we already have decent HTTP clone access.
    - jacobvosmaer 11y ago
      Hmm I was wrong, the current version of the built-in Git daemon has an --access-hook option so no problem there. https://github.com/git/git/blob/56f37fda511e1615dc6df86c68f3b841711a7828/daemon.c#L291 https://github.com/git/git/blob/56f37fda511e1615dc6df86c68f3... The question is then: how badly do people want this and how many admins out there are willing to poke an extra hole in their firewall just for this protocol.
GhotiFish 11y ago
I literally just spun up an instance of gitlab today. I'm regretting using bitnami's version though... all their documentation is out of date, and they change a lot of defaults and break things. It's been an unpleasant experience. That said. A fresh instance of bitbucket hosted close to our office vs bitbucket? No contest.
- sytse 11y ago
  Thanks for trying GitLab! We're not happy with the Bitnami version, from https://about.gitlab.com/installation/ https://about.gitlab.com/installation/ "One-click installers are frequently out of date and might not contain our Omnibus packages. An example of this are the Bitnami packages in the past couldn't be updated and are now much harder to update than the Omnibus packages. We advise to not use one-click installers but instead start an vanilla Ubuntu 14.04 instance and use the recommended Omnibus package installation. This is almost as quick as a one-click install and you're sure of the latest version and easy upgrades."
  - GhotiFish 11y ago
    OH GOD you mentioned them explicitly in the install page! That's gonna look bad for me :$
    - sytse 11y ago
      No problem :)
  - vetinari 11y ago
    While the Omnibus packages are nice, focusing on them and neglect the gitlab-only packages makes life harder for other group of people. Some of us want to reuse the pgsql/redis instances we are already running. For example, we already have pgsql for redmine and other uses, we don't want another, separate instance. Too bad that standalone packages working with system supplied (debian, centos) packages are non-existent.
    - 11y ago
giancarlostoro 11y ago
Thank you for the post, it is always great reading performance reports they make you think twice next time you develop software and question if you got any "code smells" to work on in your own code as well.
jab213 11y ago
Been using gitlab for a while and its mostly good. The only thing that is letting it down for us is the code review tools which are not as good as some of the other dedicated tools on the market. The big gripes are: * the ability to assign a review to multiple users * set a number of reviewers * Multiline comments * review publishing * Review versions Not really noticed any problems with speed but its good to see that they are working on it anyway :)
- sytse 11y ago
  Thanks for your feedback. Have you seen the merge request approvers feature in GitLab EE/.com https://about.gitlab.com/2015/06/16/feature-highlight-approve-merge-request/ https://about.gitlab.com/2015/06/16/feature-highlight-approv... ? I would love to see multiline comments too! See https://gitlab.com/gitlab-org/gitlab-ce/issues/4143 https://gitlab.com/gitlab-org/gitlab-ce/issues/4143 Review publishing, do you mean https://gitlab.com/gitlab-org/gitlab-ce/issues/3364 https://gitlab.com/gitlab-org/gitlab-ce/issues/3364 ? Review version, what do you mean with this?
  - jab214 11y ago
    "Thanks for your feedback. Have you seen the merge request approvers feature in GitLab EE/.com https://about.gitlab.com/2015/06/16/feature-highlight-approv.. https://about.gitlab.com/2015/06/16/feature-highlight-approv.... ?" Yes saw that. Currently evaluating various git servers as its something we use a lot. But would really want to assign the review to particular (or a group of) people i.e. the more experienced people in a team. "I would love to see multiline comments too! See https://gitlab.com/gitlab-org/gitlab-ce/issues/4143" https://gitlab.com/gitlab-org/gitlab-ce/issues/4143" Cool. Didn't see that - will add my vote if I can. "Review publishing, do you mean https://gitlab.com/gitlab-org/gitlab-ce/issues/3364 https://gitlab.com/gitlab-org/gitlab-ce/issues/3364 ?" Basically yes. "Review version, what do you mean with this?" Thinking about it not sure that this is really necessary in the git world. Back in svn days seeing the diffs versioned was handy but with merge request you can see the commit history so its not necessary.
    - 11y ago
Keats 11y ago
Unrelated but I signed up on the hosted gitlab and I imported some repos from github (worked flawlessly and was very easy) but I got stuck at the .travis.yml -> .gitlab-ci.yml since the documentation on the CI is not too great (or I didn't it). Is there a plan to convert or support travis ci config? If not, would it be possible to get a repo example for each language. I think i've seen something similar but there was only 1-2 examples
- sytse 11y ago
  I'm glad to hear the import went smooth and I'm sorry to hear you got stuck on the CI yml format. We're working on more examples there and are planning two blogposts about this next week. For our blogpost planning see https://gitlab.com/gitlab-com/blog-posts/issues https://gitlab.com/gitlab-com/blog-posts/issues
skywritergr 11y ago
We've been using Gitlab with my team for the past year or so. Moved from corporate Github to it. It's really nice and almost flawless so far. When i tried to install it, at some point, to a VPS of mine i had several issues with ruby packages etc. I might give it another go, it seems like the latest versions have been improved significantly.
- sytse 11y ago
  Glad yo hear the experience of your team has been good. Please consider using our default Omnibus packages to prevent problems with installing ruby.

215 points by sytse 11y ago

sytse 11y ago

Previous discussion earlier today in https://news.ycombinator.com/item?id=11173254 https://news.ycombinator.com/item?id=11173254

deleted 11y ago

[deleted]

sytse 11y ago
Yes, it is much faster, see https://about.gitlab.com/2016/02/22/gitlab-8-5-released/ https://about.gitlab.com/2016/02/22/gitlab-8-5-released/ and https://gitlab.com/gitlab-com/operations/42 https://gitlab.com/gitlab-com/operations/42

sergiotapia 11y ago

Where can I find this tool Sherlock you guys mention? It sounds interesting.

YorickPeterse 11y ago
It's built into GitLab itself, it's not an external tool you can install without using GitLab. Some information about it can be found at http://doc.gitlab.com/ce/development/profiling.html#sherlock http://doc.gitlab.com/ce/development/profiling.html#sherlock.

dogweather 11y ago

Awesome - really happy to see improvements here. It looks like the front-end can use some attention too: PageSpeed Score is only C (79%): https://gtmetrix.com/reports/gitlab.com/57l7nHlc https://gtmetrix.com/reports/gitlab.com/57l7nHlc This is the most valuable measurement, IMO. Server response time is just a proxy for end-user experience.

sytse 11y ago
Thanks for caring, we'll have a look at this.
jakecodes 11y ago
Thanks for taking a look at our frontend speed. I've created a few issues so we can take a look at this. https://gitlab.com/gitlab-org/gitlab-ce/issues/13820 https://gitlab.com/gitlab-org/gitlab-ce/issues/13820

neom 11y ago

Is the GitLab CEO here?

gkoberger 11y ago
CEO is the poster, 'sytse'.
- sytse 11y ago
  lol
sciurus 11y ago
In case you're like me and didn't get the joke at first: https://www.google.com/search?q=%22gitlab+ceo+here%22&ie=utf-8&oe=utf-8 https://www.google.com/search?q=%22gitlab+ceo+here%22&ie=utf...

OSButler 11y ago

I was running GitLab on a VPS with specs close to the lower end of the recommended requirements and its performance would gradually slow down over the course of the day until pages seemed to never finish loading. Restarting gitlab got it responding properly again, but the whole process would repeat over and over again, so that it had to be restarted every couple of hours just to be able to use it. I then switched providers, which also came with more system resources, and GitLab was working without any issues right away. It could have been caused by an oversold node at the previous host, or the slightly higher system specs at the new host made the difference, but I've been running GitLab on that new server for a couple of months now without experiencing those slowdowns again. If you've noticed a gradual slowdown to the point of pages getting stuck on loading indefinitely, then try to up the system specs a bit, as it could make quite the difference.

sytse 11y ago
What can also help is running the Omnibus package that restarts processes that start using too much memory automatically.
- parfe 11y ago
  The Ruby on rails solution to memory leaks probably shouldn't be the goto remedy.
  - rgbrenner 11y ago
    gitlab IS a ruby on rails app Why wouldn't a ruby on rails solution be right for a ruby on rails app?
    - prawnsalad 11y ago
      If the ruby on rails solution to memory leaks is to restart, then it's far better to actually get it fixed to not leak memory in the first place. Restarting is just ignoring the issue.
  - sytse 11y ago

victor9000 11y ago

I finally decided to give GitLab a shot, and so far things are going pretty well. You can make an account with your GitHub login, and you can migrate projects with just one click. From there I just had to set a password for my account, and now I'm pushing commits with no trouble at all. The whole process took just a few minutes, pretty cool!

sytse 11y ago
Glad to hear that!

9erdelta 11y ago

This is great! I moved my team's repos to GitLab last year and I love it. Pricing is better, there's more granularity in assigning team member roles, and the mobile app is good too.

sytse 11y ago
Great to hear that Matt!

shaunol 11y ago

Kudos to the GitLab team. I've been running it for 12 months now on premises. The Omnibus package makes the instance unbelievable easy to install and upgrade, so there's no excuse not to run the latest versions. I've just been watching my instance get better and better over those 12 months, better UI, more features and better performance. I really love the software and can't talk it up enough.

sytse 11y ago
Thanks shaunol, glad to hear that!

shadeslayer 11y ago

I've been using gitlab for the past few weeks to host a couple of repositories, it's awesome so far since it's the only solution that allowed me to mirror a repo AND create my own branch on the mirrored repository. The only complaint I really have is the fact that I have to append a ".git" to the http url to clone the repository. Alternatively, it'd be nice to have a git:// read only url too.

sytse 11y ago
Feel free to open an issue both for the http clone without .git, it seems interesting but I'm not sure how complicated it is. Not sure about the git:// read only url, I don't oversee the consequences.
- mintplant 11y ago
  git has a default user agent that looks like git/2.0.0, so you could detect that and redirect. I believe this is how GitHub makes this work.
- jacobvosmaer 11y ago
  The git:// protocol requires its own daemon (which is bundled with Git) running on port 9418 https://git-scm.com/book/en/v2/Git-on-the-Server-The-Protocols#The-Git-Protocol https://git-scm.com/book/en/v2/Git-on-the-Server-The-Protoco... . The standard Git daemon is unsuitable for GitLab I think because of the way it manages access control. https://git-scm.com/book/en/v2/Git-on-the-Server-Git-Daemon https://git-scm.com/book/en/v2/Git-on-the-Server-Git-Daemon For this to work in GitLab we would need to find or write an alternative implementation of the protocol. Sounds like a lot of work to me when we already have decent HTTP clone access.
  - jacobvosmaer 11y ago
    Hmm I was wrong, the current version of the built-in Git daemon has an --access-hook option so no problem there. https://github.com/git/git/blob/56f37fda511e1615dc6df86c68f3b841711a7828/daemon.c#L291 https://github.com/git/git/blob/56f37fda511e1615dc6df86c68f3... The question is then: how badly do people want this and how many admins out there are willing to poke an extra hole in their firewall just for this protocol.

GhotiFish 11y ago

I literally just spun up an instance of gitlab today. I'm regretting using bitnami's version though... all their documentation is out of date, and they change a lot of defaults and break things. It's been an unpleasant experience. That said. A fresh instance of bitbucket hosted close to our office vs bitbucket? No contest.

sytse 11y ago
Thanks for trying GitLab! We're not happy with the Bitnami version, from https://about.gitlab.com/installation/ https://about.gitlab.com/installation/ "One-click installers are frequently out of date and might not contain our Omnibus packages. An example of this are the Bitnami packages in the past couldn't be updated and are now much harder to update than the Omnibus packages. We advise to not use one-click installers but instead start an vanilla Ubuntu 14.04 instance and use the recommended Omnibus package installation. This is almost as quick as a one-click install and you're sure of the latest version and easy upgrades."
- GhotiFish 11y ago
  OH GOD you mentioned them explicitly in the install page! That's gonna look bad for me :$
  - sytse 11y ago
    No problem :)
- vetinari 11y ago
  While the Omnibus packages are nice, focusing on them and neglect the gitlab-only packages makes life harder for other group of people. Some of us want to reuse the pgsql/redis instances we are already running. For example, we already have pgsql for redmine and other uses, we don't want another, separate instance. Too bad that standalone packages working with system supplied (debian, centos) packages are non-existent.
  - 11y ago

giancarlostoro 11y ago

Thank you for the post, it is always great reading performance reports they make you think twice next time you develop software and question if you got any "code smells" to work on in your own code as well.

jab213 11y ago

Been using gitlab for a while and its mostly good. The only thing that is letting it down for us is the code review tools which are not as good as some of the other dedicated tools on the market. The big gripes are: * the ability to assign a review to multiple users * set a number of reviewers * Multiline comments * review publishing * Review versions Not really noticed any problems with speed but its good to see that they are working on it anyway :)

sytse 11y ago
Thanks for your feedback. Have you seen the merge request approvers feature in GitLab EE/.com https://about.gitlab.com/2015/06/16/feature-highlight-approve-merge-request/ https://about.gitlab.com/2015/06/16/feature-highlight-approv... ? I would love to see multiline comments too! See https://gitlab.com/gitlab-org/gitlab-ce/issues/4143 https://gitlab.com/gitlab-org/gitlab-ce/issues/4143 Review publishing, do you mean https://gitlab.com/gitlab-org/gitlab-ce/issues/3364 https://gitlab.com/gitlab-org/gitlab-ce/issues/3364 ? Review version, what do you mean with this?
- jab214 11y ago
  "Thanks for your feedback. Have you seen the merge request approvers feature in GitLab EE/.com https://about.gitlab.com/2015/06/16/feature-highlight-approv.. https://about.gitlab.com/2015/06/16/feature-highlight-approv.... ?" Yes saw that. Currently evaluating various git servers as its something we use a lot. But would really want to assign the review to particular (or a group of) people i.e. the more experienced people in a team. "I would love to see multiline comments too! See https://gitlab.com/gitlab-org/gitlab-ce/issues/4143" https://gitlab.com/gitlab-org/gitlab-ce/issues/4143" Cool. Didn't see that - will add my vote if I can. "Review publishing, do you mean https://gitlab.com/gitlab-org/gitlab-ce/issues/3364 https://gitlab.com/gitlab-org/gitlab-ce/issues/3364 ?" Basically yes. "Review version, what do you mean with this?" Thinking about it not sure that this is really necessary in the git world. Back in svn days seeing the diffs versioned was handy but with merge request you can see the commit history so its not necessary.
  - 11y ago

Keats 11y ago

Unrelated but I signed up on the hosted gitlab and I imported some repos from github (worked flawlessly and was very easy) but I got stuck at the .travis.yml -> .gitlab-ci.yml since the documentation on the CI is not too great (or I didn't it). Is there a plan to convert or support travis ci config? If not, would it be possible to get a repo example for each language. I think i've seen something similar but there was only 1-2 examples

sytse 11y ago
I'm glad to hear the import went smooth and I'm sorry to hear you got stuck on the CI yml format. We're working on more examples there and are planning two blogposts about this next week. For our blogpost planning see https://gitlab.com/gitlab-com/blog-posts/issues https://gitlab.com/gitlab-com/blog-posts/issues

skywritergr 11y ago

We've been using Gitlab with my team for the past year or so. Moved from corporate Github to it. It's really nice and almost flawless so far. When i tried to install it, at some point, to a VPS of mine i had several issues with ruby packages etc. I might give it another go, it seems like the latest versions have been improved significantly.

sytse 11y ago
Glad yo hear the experience of your team has been good. Please consider using our default Omnibus packages to prevent problems with installing ruby.