7 ms·

Critical Security Release for GitLab 8.2 through 8.7

95 points by iMerNibor 10y ago

beefhash 10y ago
Is it just me or has gitlab issued a lot of critical security releases lately? I'm not sure if this is because they're particularly open about these things or because their product might be particularly insecure.
- JBReefer 10y ago
  Having worked in a large set of codebases as a consultant, I'm pretty sure it's the former. The code that runs the world is mostly garbage (the proprietary stuff that is, because they can hide it)
  - nitrogen 10y ago
    Similarly, never count on a compromise of your data to get reported unless local law absolutely requires it, and someone who cares notices. If your data was compromised in a way not covered by local law, companies will hide the compromise.
- dalanmiller 10y ago
  Other commentor already nailed it I think. How many critical security issues do you think Github has had that you've never heard of? Not knocking Github, it's just the nature of extreme open source I think.
  - danielsamuels 10y ago
    Github isn't open source.
    - ctr 10y ago
      But GitLab is and that's what the comment was about.
- CameronBanga 10y ago
  Finding, fixing, and openly discussing software issues makes me more confident in GitLab, not less. I see this as a positive thing.
  - sytse 10y ago
    Thanks! We're working on a postmortem blog post about our process for releasing these fixes.
- JamesSwift 10y ago
  Your comment is weirdly similar to this other comment that I saw in the initial announcement: https://news.ycombinator.com/item?id=11593362 https://news.ycombinator.com/item?id=11593362
  - unethical_ban 10y ago
    I had to check that it wasn't the same user. I know groupthink is a thing, but the cynic in me wondered if this comment wasn't an upvote grab.
  - sdesol 10y ago
    Since we are doing deja vu, I uploaded some new metrics that better highlights how insane GitLab's churn rate is. http://imgur.com/a/4uaSR http://imgur.com/a/4uaSR What's really interesting is the number of contributors.
    - StavrosK 10y ago
      What does this show, exactly? I'm not familiar with this tool and there are no labels.
      - sdesol 10y ago
        It basically shows how much has changed in GitLab's master branch in the last 30 days. The main metrics is cumulative code churn (lines added, changed, and deleted that does not involve comment or blank lines). In the first picture it shows the churn was about 24,000 if you don't include merge commits (nomerge:true) and if you ignore changes that were the result of adding/deleting files (action:M). The numbers with the avatars are basically. First column is number of commits. Second column is cumulative code churn. Third column is percentage of churn that does not involve blank/comment lines. The second and third pictures shows the code churn as grouped by top level directories. In GitLab's case the most churn occurred in the app directory. And if you drill down to app/assets/javascripts, you'll see the following: http://imgur.com/B2GE1yk http://imgur.com/B2GE1yk The charts and metrics basically shows GitLab's code base is changing a lot. This is the churn for Gogs in the last 30 days. http://imgur.com/ZmzyMsz http://imgur.com/ZmzyMsz And I guess the question is, is the high rate of change a contributing factor for increased security issues? Statistically speaking, the more code, the more chances for something being missed during code reviews. It's important to note the metrics isn't saying the quality is bad. It's just saying a lot is changing.
- neandrake 10y ago
  You should also factor in their software model. GitLab has a free install anyone can run locally without having to register. A competitor like GitHub has enterprise install but otherwise most public traffic is on their own hosted install. If there's a security issue they likely fix their hosted install and inform their enterprise customers only -- as well they probably shouldn't make it public otherwise it could be considered some level of exposure to their customers. Conversely GitLab essentially needs to publicize this to maintain the trust of their customers.
- viraptor 10y ago
  There are other reasons too. Once someone finds an issue in an open-source project, other people realise it's not super secure and start to dig deeper. Same thing happened with image decoders (libpng), asn1 parsers, syscalls fuzzing, digging into openssl, etc. Bugs in big products and common protocols come in waves.
- ashitlerferad 10y ago
  I think that makes them more trustworthy. If some open source software has never seen a CVE, then that is because no-one actually looked at it in depth enough. All software has bugs and security issues.
- educar 10y ago
  Not been tracking GitLab closely but how many critical issues have there been in the last 6 months?
kentonv 10y ago
Gitlab can be run on Sandstorm.io (of which I am tech lead / co-founder). Sandstorm claims to mitigate most vulnerabilities in apps: https://docs.sandstorm.io/en/latest/using/security-non-events/ https://docs.sandstorm.io/en/latest/using/security-non-event... Let's see how it scores here... For background, on Sandstorm, each Gitlab project is placed in a separate grain (container), isolated from all others. In order to communicate with a grain at all, you must have been granted some level of access to it by its owner -- Sandstorm does not let you send requests to private grains to which you haven't been given access. So, private Gitlab repos hosted on Sandstorm are basically not vulnerable to any vulnerability. Of course, Gitlab is the kind of thing you might intentionally make public to all, e.g. to host an open source project. Therefore, it makes sense to analyze whether a public repository would be exploitable. Privilege escalation via "impersonate" feature On Sandstorm, authentication is handled by Sandstorm. The app receives an unspoofable header indicating which user the request came from, and what permissions they have. A well-written app uses this header on every request to authenticate the user. Unfortunately, our Gitlab package currently uses this information only when a session first opens, then relies on the session cookie going forward. This pattern is sometimes used as a "hack" on Sandstorm to more easily integrate with existing login code designed to do upfront / one-time authentication. As such, public Gitlab repositories hosted on Sandstorm would be vulnerable. Had Gitlab on Sandstorm been implemented "properly", it would not be vulnerable. That said, the "impersonate user" feature would not have worked at all. That's probably for the best: a feature like this really ought to be implemented by Sandstorm itself, which would have the ability to implement it (securely) across all apps at once, rather than have each app implement its own version. Somewhat embarrassingly, the Sandstorm package of Gitlab actually predates this feature being added, therefore Gitlab instances on Sandstorm today actually aren't vulnerable. (Generally, if the upstream app author does not directly maintain the Sandstorm package, then the Sandstorm package will tend to fall behind. This should get better as Sandstorm gains popularity and upstream authors target it explicitly.) Privilege escalation via notes API Privilege escalation via project webhook API Information disclosure via project labels Information disclosure via new merge request page These vulnerabilities allow a user to manipulate a private project to which they aren't supposed to have access. On Sandstorm, a private project would live in its own grain, and if you hadn't been given access then Sandstorm would deny you the ability to talk to the grain at all. Therefore, these cannot be exploited. XSS vulnerability via branch and tag names XSS vulnerability via custom issue tracker URL XSS vulnerability via label drop-down These vulnerabilities require that you have write access to one project on the server in order to launch an attack. On Sandstorm, since every repository is its own instance, the attack would be limited to the repository on which you have write access -- you would not be able to use this attack to damage someone else's Gitlab repository on which you lack write access. XSS vulnerability via window.opener This is a subtle phishing issue that is very widespread. However, it mostly doesn't work when the opener is a sandstorm app: the app lives inside an iframe which is prohibited by Content-Security-Policy from browsing away from the server. Information disclosure via milestone API Information disclosure via snippet API A public project hosted on Sandstorm would be vulnerable to these (leaking confidential issues attached to public milestones, and leaking private snippets attached to a public project). Sandstorm can only enforce access control at the grain level; anything finer than that is up to the app, and is subject to app bugs. I generally recommend that Sandstorm users try to put confidential data in separate grains from public data -- e.g. creating a separate issue tracker for confidential issues. (The Sandstorm packaging again predates the milestone bug's introduction, though may be affected by the snippet issue.) We will update the Gitlab package tomorrow. Once an update is pushed, every Sandstorm user will receive a notification within 24 hours and can apply the update with one click. Conclusion: Sandstorm mitigated 8/11 issues by design, 2/11 by accident, and is vulnerable to 1/11. The biggest issue was only mitigated by accident in this case, although a well-behaved Sandstorm app normally wouldn't have this kind of issue by design. Overall, though, I'm disappointed in Sandstorm's performance here -- it's much worse than the usual 95% mitigation rate.
gramakri 10y ago
Thanks for the release! We updated the cloudron.io app to use 8.7.1 we well. Cloudron users will get GitLab auto-updated tonight.
dewiz 10y ago
Note for admins using Apache 2.4, regarding the manual quick fix described, take into consideration the Apache auth config changes described here https://httpd.apache.org/docs/current/upgrading.html https://httpd.apache.org/docs/current/upgrading.html

95 points by iMerNibor 10y ago

Is it just me or has gitlab issued a lot of critical security releases lately? I'm not sure if this is because they're particularly open about these things or because their product might be particularly insecure.

JBReefer 10y ago
Having worked in a large set of codebases as a consultant, I'm pretty sure it's the former. The code that runs the world is mostly garbage (the proprietary stuff that is, because they can hide it)
- nitrogen 10y ago
  Similarly, never count on a compromise of your data to get reported unless local law absolutely requires it, and someone who cares notices. If your data was compromised in a way not covered by local law, companies will hide the compromise.
dalanmiller 10y ago
Other commentor already nailed it I think. How many critical security issues do you think Github has had that you've never heard of? Not knocking Github, it's just the nature of extreme open source I think.
- danielsamuels 10y ago
  Github isn't open source.
  - ctr 10y ago
    But GitLab is and that's what the comment was about.
CameronBanga 10y ago
Finding, fixing, and openly discussing software issues makes me more confident in GitLab, not less. I see this as a positive thing.
- sytse 10y ago
  Thanks! We're working on a postmortem blog post about our process for releasing these fixes.
JamesSwift 10y ago
Your comment is weirdly similar to this other comment that I saw in the initial announcement: https://news.ycombinator.com/item?id=11593362 https://news.ycombinator.com/item?id=11593362
- unethical_ban 10y ago
  I had to check that it wasn't the same user. I know groupthink is a thing, but the cynic in me wondered if this comment wasn't an upvote grab.
- sdesol 10y ago
  Since we are doing deja vu, I uploaded some new metrics that better highlights how insane GitLab's churn rate is. http://imgur.com/a/4uaSR http://imgur.com/a/4uaSR What's really interesting is the number of contributors.
  - StavrosK 10y ago
    What does this show, exactly? I'm not familiar with this tool and there are no labels.
    - sdesol 10y ago
      It basically shows how much has changed in GitLab's master branch in the last 30 days. The main metrics is cumulative code churn (lines added, changed, and deleted that does not involve comment or blank lines). In the first picture it shows the churn was about 24,000 if you don't include merge commits (nomerge:true) and if you ignore changes that were the result of adding/deleting files (action:M). The numbers with the avatars are basically. First column is number of commits. Second column is cumulative code churn. Third column is percentage of churn that does not involve blank/comment lines. The second and third pictures shows the code churn as grouped by top level directories. In GitLab's case the most churn occurred in the app directory. And if you drill down to app/assets/javascripts, you'll see the following: http://imgur.com/B2GE1yk http://imgur.com/B2GE1yk The charts and metrics basically shows GitLab's code base is changing a lot. This is the churn for Gogs in the last 30 days. http://imgur.com/ZmzyMsz http://imgur.com/ZmzyMsz And I guess the question is, is the high rate of change a contributing factor for increased security issues? Statistically speaking, the more code, the more chances for something being missed during code reviews. It's important to note the metrics isn't saying the quality is bad. It's just saying a lot is changing.
neandrake 10y ago
You should also factor in their software model. GitLab has a free install anyone can run locally without having to register. A competitor like GitHub has enterprise install but otherwise most public traffic is on their own hosted install. If there's a security issue they likely fix their hosted install and inform their enterprise customers only -- as well they probably shouldn't make it public otherwise it could be considered some level of exposure to their customers. Conversely GitLab essentially needs to publicize this to maintain the trust of their customers.
viraptor 10y ago
There are other reasons too. Once someone finds an issue in an open-source project, other people realise it's not super secure and start to dig deeper. Same thing happened with image decoders (libpng), asn1 parsers, syscalls fuzzing, digging into openssl, etc. Bugs in big products and common protocols come in waves.
ashitlerferad 10y ago
I think that makes them more trustworthy. If some open source software has never seen a CVE, then that is because no-one actually looked at it in depth enough. All software has bugs and security issues.
educar 10y ago
Not been tracking GitLab closely but how many critical issues have there been in the last 6 months?

kentonv 10y ago

Gitlab can be run on Sandstorm.io (of which I am tech lead / co-founder). Sandstorm claims to mitigate most vulnerabilities in apps: https://docs.sandstorm.io/en/latest/using/security-non-events/ https://docs.sandstorm.io/en/latest/using/security-non-event... Let's see how it scores here... For background, on Sandstorm, each Gitlab project is placed in a separate grain (container), isolated from all others. In order to communicate with a grain at all, you must have been granted some level of access to it by its owner -- Sandstorm does not let you send requests to private grains to which you haven't been given access. So, private Gitlab repos hosted on Sandstorm are basically not vulnerable to any vulnerability. Of course, Gitlab is the kind of thing you might intentionally make public to all, e.g. to host an open source project. Therefore, it makes sense to analyze whether a public repository would be exploitable. Privilege escalation via "impersonate" feature On Sandstorm, authentication is handled by Sandstorm. The app receives an unspoofable header indicating which user the request came from, and what permissions they have. A well-written app uses this header on every request to authenticate the user. Unfortunately, our Gitlab package currently uses this information only when a session first opens, then relies on the session cookie going forward. This pattern is sometimes used as a "hack" on Sandstorm to more easily integrate with existing login code designed to do upfront / one-time authentication. As such, public Gitlab repositories hosted on Sandstorm would be vulnerable. Had Gitlab on Sandstorm been implemented "properly", it would not be vulnerable. That said, the "impersonate user" feature would not have worked at all. That's probably for the best: a feature like this really ought to be implemented by Sandstorm itself, which would have the ability to implement it (securely) across all apps at once, rather than have each app implement its own version. Somewhat embarrassingly, the Sandstorm package of Gitlab actually predates this feature being added, therefore Gitlab instances on Sandstorm today actually aren't vulnerable. (Generally, if the upstream app author does not directly maintain the Sandstorm package, then the Sandstorm package will tend to fall behind. This should get better as Sandstorm gains popularity and upstream authors target it explicitly.) Privilege escalation via notes API Privilege escalation via project webhook API Information disclosure via project labels Information disclosure via new merge request page These vulnerabilities allow a user to manipulate a private project to which they aren't supposed to have access. On Sandstorm, a private project would live in its own grain, and if you hadn't been given access then Sandstorm would deny you the ability to talk to the grain at all. Therefore, these cannot be exploited. XSS vulnerability via branch and tag names XSS vulnerability via custom issue tracker URL XSS vulnerability via label drop-down These vulnerabilities require that you have write access to one project on the server in order to launch an attack. On Sandstorm, since every repository is its own instance, the attack would be limited to the repository on which you have write access -- you would not be able to use this attack to damage someone else's Gitlab repository on which you lack write access. XSS vulnerability via window.opener This is a subtle phishing issue that is very widespread. However, it mostly doesn't work when the opener is a sandstorm app: the app lives inside an iframe which is prohibited by Content-Security-Policy from browsing away from the server. Information disclosure via milestone API Information disclosure via snippet API A public project hosted on Sandstorm would be vulnerable to these (leaking confidential issues attached to public milestones, and leaking private snippets attached to a public project). Sandstorm can only enforce access control at the grain level; anything finer than that is up to the app, and is subject to app bugs. I generally recommend that Sandstorm users try to put confidential data in separate grains from public data -- e.g. creating a separate issue tracker for confidential issues. (The Sandstorm packaging again predates the milestone bug's introduction, though may be affected by the snippet issue.) We will update the Gitlab package tomorrow. Once an update is pushed, every Sandstorm user will receive a notification within 24 hours and can apply the update with one click. Conclusion: Sandstorm mitigated 8/11 issues by design, 2/11 by accident, and is vulnerable to 1/11. The biggest issue was only mitigated by accident in this case, although a well-behaved Sandstorm app normally wouldn't have this kind of issue by design. Overall, though, I'm disappointed in Sandstorm's performance here -- it's much worse than the usual 95% mitigation rate.

gramakri 10y ago

Thanks for the release! We updated the cloudron.io app to use 8.7.1 we well. Cloudron users will get GitLab auto-updated tonight.

dewiz 10y ago

Note for admins using Apache 2.4, regarding the manual quick fix described, take into consideration the Apache auth config changes described here https://httpd.apache.org/docs/current/upgrading.html https://httpd.apache.org/docs/current/upgrading.html