7 ms·

Rob Zuber, CTO of CircleCI here. We take security very seriously and are taking a deep look at the issues Kevin raised. We'll save additional commenting until

by z00b 9y ago

Rob Zuber, CTO of CircleCI here. We take security very seriously and are taking a deep look at the issues Kevin raised. We'll save additional commenting until we have gathered more information. In the meantime, our security policy and steps for reporting issues are here: https://circleci.com/security/ https://circleci.com/security/ and we'd like to ask the community to please use our outlined methods for reporting potential security issues so we can keep CircleCI as safe as possible for everyone.

caust1c 9y ago
Who's Ryan?
- z00b 9y ago
  Welp. Wanted to write quickly and made a stupid mistake. Sorry about that Kevin!
kevinburke 9y ago
OP here. Thanks for responding promptly. To be clear, letting third party JS run in a trusted environment like a dashboard is an industry wide problem. If we assume CircleCI is the only bad actor we're kind of missing the point of the exercise. CircleCI is a notable example, due to the fact it hosts source code and secrets for so many different companies and loads so many third party scripts in its dashboard context. But they're not unique in this regard. I hope everyone reading this is reconsidering the scripts that have access to their dashboard and pushing for changes at their company. If CircleCI was immediately vulnerable to injection via the scripts above I would have used the private disclosure route and I encourage others to as well. But I don't know that there is a "vulnerability" here so much as a discussion that we need to have about what and how much third party code we let run in a trusted context. I wrote a little more in the thread below, https://news.ycombinator.com/item?id=15442988 https://news.ycombinator.com/item?id=15442988.
- z00b 9y ago
  Thanks Kevin. We really do appreciate the discussion and, as you would imagine, are rethinking some of our approaches in line with the issues you've raised.
  - Posibyte 9y ago
    The real thanks should go to you Rob. As it's stated, it's an industry wide problem and the real guts come from the companies that take the issue seriously; the ones who come out, hold it close to their own, admit they were wrong, and work to improve. This says a lot about you and your company, and as a customer it makes me proud that you're part of our mobile pipeline. As a customer, Rob, what would be a good avenue to notify you of issues like this in the future? Not every company wants to be front and center of what is essentially the news hub of tech people like HN for issues, so is there some way we can get with you guys directly to straighten security issues out, as well as some assurances as how you'll handle it along the way? Thanks for your attention to the issue :)
    - z00b 9y ago
      Hi. You can always email our security team at security@circleci.com Our gpp key is available on our site https://circleci.com/security/ https://circleci.com/security/ if needed. You can also email me directly at rob@ if you have an immediate question.
  - remcob 9y ago
    In the interest of said discussion, I would appreciate a write-up of the rationale behind the approach chosen, when you are ready. It could help others that are in a similar situation. Perhaps inspire a better practice in the industry.
    - ssemmaprise 9y ago
      Hi there, we wrote up our thoughts + reply here: https://discuss.circleci.com/t/circleci-response-to-kevin-burke-s-blog-post-about-third-party-services/17276 https://discuss.circleci.com/t/circleci-response-to-kevin-bu... Thanks all for your thoughts + comments.
atomi 9y ago
What serious business hosts their proprietary, mission critical code on someone else's computers? Isn't that like Coke or KFC putting their original recipe on Google Drive? Edit: Okay, I get it. People like convenience over security. I'll stick to self-hosting a Drone-CI instance and only deploying binaries.
- Phlogistique 9y ago
  Just in case this is a serious question: Most of them.
- timdorr 9y ago
  Literally everyone on AWS, for instance. Except Amazon, of course.
oliwarner 9y ago
I'm not sure private disclosure is appropriate here. This isn't the same as a browser vulnerability where some kid could hack a load of people before they patched. One of these supposedly trusted companies could attack your customers... But they could already. They know their scripts get loaded into all sorts of inappropriate environments. Getting people notified, getting credentials changed. That's the paramount concern. You can rub PR lotion into an in-depth audit later on.
- ejcx 9y ago
  This reaction is nonsense. These are all vendors that circle pays. It's an industry wide bad practice and a risk, but forcing password changes or notifying people is quite frankly ridiculous.
  - oliwarner 9y ago
    I didn't suggest forcing anything... But just because other people do it doesn't make it better. Many of their clients may treat this as they would an actual breach. It's somebody they haven't vetted having potentially complete access to their development chain and production secrets. They won't know until they look. They won't look until they're told. And what's CircleCI paying a for this breach got to do with the price of fish?! Say you hired me and gave me full access to everything in your business. Then one day I turn around and tell you my extended family, my friends, my dog walker and my cleaner have all also had access to that data. No big problem eh?
    - ejcx 9y ago
      There is absolutely no breach here. Absolutely not. Suggesting so is ridiculous. The word breach is a very special one and this is not it.
      - oliwarner 9y ago
        I think you're the one being more reckless with language here. But please, what does "breach" mean to you? I count it as "inappropriate and unauthorised access to data". Where "access" is potential, not necessarily actual unless you can absolutely prove there was no access. These third parties have had access to sensitive data they shouldn't. That's a breach in my book. Neither you or Circle CI even can say this hasn't lead to current or past third parties —or their rogue developers, or people who have hacked them— gaining source access or customer data from Circle CI users. Why? You simply don't know what was running at any given point. Auditing and sub-resource integrity would help in the future, but it's too late. Unknown people have had access. Only the Circle CI users will know what ramifications that could have on them. If your argument is anything more than a redefinition of "breach", please explain why you're being so nonchalant (and why you think I'm being "ridiculous") about this.
deleted 9y ago
[deleted]
_Codemonkeyism 9y ago
"We take security very seriously" No you don't.
- ejcx 9y ago
  This reaction is ridiculous as well. Completely inappropriate comment. This is from the person who's website codemonkeyism is running on the same host as some german version of AirBNB. It's so sad to see people without a clue when it comes to security lambasting people who are writing software and are actually doing a pretty good job.
- dang 9y ago
  This breaks the HN guidelines, which require you to remain civil. We ban users who can't or won't do that, so please read https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html and fix this in the future.
bsimpson 9y ago
Is there any way to limit what access CircleCI has to my GitHub account/orgs? GitHub says you have "Full control of private repositories," but it seems like all you need is read access + a push webhook.
- kevinburke 9y ago
  IIRC Circle needs to tell Github whether the build succeeded or failed. It would be nice if Github offered more granular permissions.
  - daxelrod 9y ago
    GitHub recently added a repo:status OAuth scope for this use case. Unfortunately, I still don't see a scope for read-only access to repo code. https://developer.github.com/apps/building-integrations/setting-up-and-registering-oauth-apps/about-scopes-for-oauth-apps/ https://developer.github.com/apps/building-integrations/sett...
  - lancefisher 9y ago
    When I need more granular access, this is how I set it up: 1) Create a new GitHub user: e.g. (circleci-builder@example.com) 2) Grant read-only access to specific repositories to the new user. 3) Configure CI to use that user.

by z00b 9y ago

caust1c 9y ago

Who's Ryan?

z00b 9y ago
Welp. Wanted to write quickly and made a stupid mistake. Sorry about that Kevin!

kevinburke 9y ago

OP here. Thanks for responding promptly. To be clear, letting third party JS run in a trusted environment like a dashboard is an industry wide problem. If we assume CircleCI is the only bad actor we're kind of missing the point of the exercise. CircleCI is a notable example, due to the fact it hosts source code and secrets for so many different companies and loads so many third party scripts in its dashboard context. But they're not unique in this regard. I hope everyone reading this is reconsidering the scripts that have access to their dashboard and pushing for changes at their company. If CircleCI was immediately vulnerable to injection via the scripts above I would have used the private disclosure route and I encourage others to as well. But I don't know that there is a "vulnerability" here so much as a discussion that we need to have about what and how much third party code we let run in a trusted context. I wrote a little more in the thread below, https://news.ycombinator.com/item?id=15442988 https://news.ycombinator.com/item?id=15442988.

z00b 9y ago
Thanks Kevin. We really do appreciate the discussion and, as you would imagine, are rethinking some of our approaches in line with the issues you've raised.
- Posibyte 9y ago
  The real thanks should go to you Rob. As it's stated, it's an industry wide problem and the real guts come from the companies that take the issue seriously; the ones who come out, hold it close to their own, admit they were wrong, and work to improve. This says a lot about you and your company, and as a customer it makes me proud that you're part of our mobile pipeline. As a customer, Rob, what would be a good avenue to notify you of issues like this in the future? Not every company wants to be front and center of what is essentially the news hub of tech people like HN for issues, so is there some way we can get with you guys directly to straighten security issues out, as well as some assurances as how you'll handle it along the way? Thanks for your attention to the issue :)
  - z00b 9y ago
    Hi. You can always email our security team at security@circleci.com Our gpp key is available on our site https://circleci.com/security/ https://circleci.com/security/ if needed. You can also email me directly at rob@ if you have an immediate question.
- remcob 9y ago
  In the interest of said discussion, I would appreciate a write-up of the rationale behind the approach chosen, when you are ready. It could help others that are in a similar situation. Perhaps inspire a better practice in the industry.
  - ssemmaprise 9y ago
    Hi there, we wrote up our thoughts + reply here: https://discuss.circleci.com/t/circleci-response-to-kevin-burke-s-blog-post-about-third-party-services/17276 https://discuss.circleci.com/t/circleci-response-to-kevin-bu... Thanks all for your thoughts + comments.

atomi 9y ago

What serious business hosts their proprietary, mission critical code on someone else's computers? Isn't that like Coke or KFC putting their original recipe on Google Drive? Edit: Okay, I get it. People like convenience over security. I'll stick to self-hosting a Drone-CI instance and only deploying binaries.

Phlogistique 9y ago
Just in case this is a serious question: Most of them.
timdorr 9y ago
Literally everyone on AWS, for instance. Except Amazon, of course.

oliwarner 9y ago

I'm not sure private disclosure is appropriate here. This isn't the same as a browser vulnerability where some kid could hack a load of people before they patched. One of these supposedly trusted companies could attack your customers... But they could already. They know their scripts get loaded into all sorts of inappropriate environments. Getting people notified, getting credentials changed. That's the paramount concern. You can rub PR lotion into an in-depth audit later on.

ejcx 9y ago
This reaction is nonsense. These are all vendors that circle pays. It's an industry wide bad practice and a risk, but forcing password changes or notifying people is quite frankly ridiculous.
- oliwarner 9y ago
  I didn't suggest forcing anything... But just because other people do it doesn't make it better. Many of their clients may treat this as they would an actual breach. It's somebody they haven't vetted having potentially complete access to their development chain and production secrets. They won't know until they look. They won't look until they're told. And what's CircleCI paying a for this breach got to do with the price of fish?! Say you hired me and gave me full access to everything in your business. Then one day I turn around and tell you my extended family, my friends, my dog walker and my cleaner have all also had access to that data. No big problem eh?
  - ejcx 9y ago
    There is absolutely no breach here. Absolutely not. Suggesting so is ridiculous. The word breach is a very special one and this is not it.
    - oliwarner 9y ago
      I think you're the one being more reckless with language here. But please, what does "breach" mean to you? I count it as "inappropriate and unauthorised access to data". Where "access" is potential, not necessarily actual unless you can absolutely prove there was no access. These third parties have had access to sensitive data they shouldn't. That's a breach in my book. Neither you or Circle CI even can say this hasn't lead to current or past third parties —or their rogue developers, or people who have hacked them— gaining source access or customer data from Circle CI users. Why? You simply don't know what was running at any given point. Auditing and sub-resource integrity would help in the future, but it's too late. Unknown people have had access. Only the Circle CI users will know what ramifications that could have on them. If your argument is anything more than a redefinition of "breach", please explain why you're being so nonchalant (and why you think I'm being "ridiculous") about this.

deleted 9y ago

[deleted]

_Codemonkeyism 9y ago

"We take security very seriously" No you don't.

ejcx 9y ago
This reaction is ridiculous as well. Completely inappropriate comment. This is from the person who's website codemonkeyism is running on the same host as some german version of AirBNB. It's so sad to see people without a clue when it comes to security lambasting people who are writing software and are actually doing a pretty good job.
dang 9y ago
This breaks the HN guidelines, which require you to remain civil. We ban users who can't or won't do that, so please read https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html and fix this in the future.

bsimpson 9y ago

Is there any way to limit what access CircleCI has to my GitHub account/orgs? GitHub says you have "Full control of private repositories," but it seems like all you need is read access + a push webhook.

kevinburke 9y ago
IIRC Circle needs to tell Github whether the build succeeded or failed. It would be nice if Github offered more granular permissions.
- daxelrod 9y ago
  GitHub recently added a repo:status OAuth scope for this use case. Unfortunately, I still don't see a scope for read-only access to repo code. https://developer.github.com/apps/building-integrations/setting-up-and-registering-oauth-apps/about-scopes-for-oauth-apps/ https://developer.github.com/apps/building-integrations/sett...
- lancefisher 9y ago
  When I need more granular access, this is how I set it up: 1) Create a new GitHub user: e.g. (circleci-builder@example.com) 2) Grant read-only access to specific repositories to the new user. 3) Configure CI to use that user.