3 ms·
> We wanted to make you aware that we are currently investigating a security incident, and that our investigation is ongoing. We will provide you updates about
by bamboozled 4y ago
> We wanted to make you aware that we are currently investigating a security incident, and that our investigation is ongoing. We will provide you updates about this incident, and our response, as they become available. At this point, we are confident that there are no unauthorized actors active in our systems; however, out of an abundance of caution, we want to ensure that all customers take certain preventative measures to protect your data as well.
Is anyone else a little annoyed by the messaging here, I read it as, "We think something bad happened to your ultra secret data, but we don't know, so we're asking teams to spend potentially hours or days fixing things while we aren't really able to tell you if your stuff was actually compromised"?
What I find more troubling is, if they don't quite know what happened, or aren't telling us, and we do the work to change everything, how do they know it won't just happen again in the next day or so and people are still accessing our systems, where is the details?
> At this point, we are confident that there are no unauthorized actors active in our systems.
Confident isn't really a good enough word to use here in my opinion. We've just blocked Circle CI from all our systems for now until we hear more, likely start to move to another build system.
I know accidents happen but this is likely the beginning of the end for our teams relationship with Circle CI. Trust has been broken.
- csomar 4y agoI can see you are bamboozled. But you should have seen the writing on the wall for CircleCI for quite some time now.
- taf2 4y agoCan elaborate where would I have seen the writing for this? What indicators did you see?
- icco 4y agoLayoffs and outages at Circle. software supply chain attacks becoming more and more popular.
- ryanisnan 4y agoOutages at CircleCI have been common since the service has been launched. Amazon is about to lay off 18k people. Circle isn't in a unique position to my understanding. Did CircleCI lay off some group or set of really important and key personnel? Software supply chain attacks affect everyone. Is there some way that Circle is more vulnerable to this type of attack?
- berniedurfee 4y agoI would imagine GitHub Actions and always maturing CI tooling baked into cloud providers are rapidly eroding the market share of dedicated services like CircleCI.
- bamboozled 4y agoAgreed, how was I supposed to know that Circle CI would lose all the secret keys? I mean I always knew it was a possibility and our team planned for it...but what are you actually talking about?
- ryanisnan 4y agoI'm curious what indicators you have that this would have been the case. This is not a comment made in snark; I'm genuinely curious, as a CircleCI user who did not see the writing on the wall.
- aequitas 4y ago> so we're asking teams to spend potentially hours or days fixing things At the risk of sounding pedantic, but this is why you have everything as IaC. These kind of changes should not cost days. It should take merely minutes or an hour tops to change all your keys. It should be trivial, for cases just like this.
- bamboozled 4y agoI fully agree, our team just had to change one set of keys, other teams didn't follow best practices and are in a bad situation. It's not Circle's fault people didn't do things propertly, but I think they just owe us a better explanation.
- tootie 4y agoYou can't use IaC to change third-party API keys. And woe unto any service that doesn't allow multiple keys because then you're looking at outages.
- aequitas 4y agoYou can, for circleci for example: https://registry.terraform.io/providers/mrolla/circleci/latest/docs/resources/context_environment_variable https://registry.terraform.io/providers/mrolla/circleci/late...
- tootie 4y agoI get that you can manage the values in Circle, but you can't actually generate the values. IE, if you have a API token to write to Salesforce, you have to go into the Salesforce admin and generate a new token. Pasting the value in the Circle UI or a terraform descriptor are not the hard part. For lots of services, you can only have one key at a time meaning that generating a new one invalidates the old one meaning you'd have to have an outage while you're pasting and deploying.