3 ms·

> The one service token and three accounts were not rotated because mistakenly it was believed they were unused. Eh? So why weren't they revoked entirely? I'm

by OJFord 3y ago

> The one service token and three accounts were not rotated because mistakenly it was believed they were unused. Eh? So why weren't they revoked entirely? I'm sure something's just unsaid there, or lost in communication or something, but as written that doesn't really make sense to me?

stepupmakeup 3y ago
Rotating could have been manual and the person in charge wanted to save time. Stress could be a factor too.
htrp 3y ago
blameless post mortem most likely Great call out too > Note that this was in no way an error on the part of AWS, Moveworks or Smartsheet. These were merely credentials which we failed to rotate.
- OJFord 3y ago
  It can still be blameless though? The 'because' makes it sound like that's a correct reason to leave it; that the only error was thinking they were unused. (i.e. that it's fine to leave them if unused, only a problem if they're used) i.e. instead of 'because they were mistakenly thought to be unused' you can say 'because they were mistakenly thought to be ok to leave as unused' (or something less awkward depending on exactly what the scenario was) and there's no more blame there? And if you really want to emphasise blamelessness you can say how your processes and training failed to sufficiently encourage least privilege, etc.
phyzome 3y ago
Betting they have a new item in their compromise runbook. :-)
- OJFord 3y ago
  No I don't think so, I do think something's just difficult to say because of what they can't say, or they just neglected to say/didn't word it well, or something. i.e. a bug in the writing, not the post mortem itself. Because if you take it exactly as it's written it's just too weird, I'm not a security expert with something to teach Cloudflare about err maybe don't leave secrets lying around that aren't actually needed for anything, that's not news to many people, and they surely have many actual security people for whom that would not even be a fizzbuzz interview question reviewing any kind of secret storage or revocation policy/procedure. And also the mentioned third-party audit.
crdrost 3y ago
I would assume that "believed" is not meant to be interpreted in an active personal sense but in a passive configuration sense. That is, I'd expect there was a flag in a database somewhere saying that those service accounts were "abandoned" or "cleaned up" or some other non-active status, but that this assertion was incorrect. Then they probably rotated all the passwords for active accounts, but skipped the inactive ones. Speaking purely about PKI and certificate revocation, because that's the only similar context that I really know about, there is generally a difference between allowing certificates to expire, vs allowing them to be marked as "no longer used", vs fully revoking them: a certificate authority needs to do absolutely nothing in the first case, can choose to either do nothing or revoke in the second case, and must actively maintain and broadcast that revocation list for the third case. When someone says "hey I accidentally clobbered that private key can I please have a new cert for this new key," you generally don't add the old cert to the revocation list because why would you.

by OJFord 3y ago

stepupmakeup 3y ago

Rotating could have been manual and the person in charge wanted to save time. Stress could be a factor too.

htrp 3y ago

blameless post mortem most likely Great call out too > Note that this was in no way an error on the part of AWS, Moveworks or Smartsheet. These were merely credentials which we failed to rotate.

OJFord 3y ago
It can still be blameless though? The 'because' makes it sound like that's a correct reason to leave it; that the only error was thinking they were unused. (i.e. that it's fine to leave them if unused, only a problem if they're used) i.e. instead of 'because they were mistakenly thought to be unused' you can say 'because they were mistakenly thought to be ok to leave as unused' (or something less awkward depending on exactly what the scenario was) and there's no more blame there? And if you really want to emphasise blamelessness you can say how your processes and training failed to sufficiently encourage least privilege, etc.

phyzome 3y ago

Betting they have a new item in their compromise runbook. :-)

OJFord 3y ago
No I don't think so, I do think something's just difficult to say because of what they can't say, or they just neglected to say/didn't word it well, or something. i.e. a bug in the writing, not the post mortem itself. Because if you take it exactly as it's written it's just too weird, I'm not a security expert with something to teach Cloudflare about err maybe don't leave secrets lying around that aren't actually needed for anything, that's not news to many people, and they surely have many actual security people for whom that would not even be a fizzbuzz interview question reviewing any kind of secret storage or revocation policy/procedure. And also the mentioned third-party audit.

crdrost 3y ago

I would assume that "believed" is not meant to be interpreted in an active personal sense but in a passive configuration sense. That is, I'd expect there was a flag in a database somewhere saying that those service accounts were "abandoned" or "cleaned up" or some other non-active status, but that this assertion was incorrect. Then they probably rotated all the passwords for active accounts, but skipped the inactive ones. Speaking purely about PKI and certificate revocation, because that's the only similar context that I really know about, there is generally a difference between allowing certificates to expire, vs allowing them to be marked as "no longer used", vs fully revoking them: a certificate authority needs to do absolutely nothing in the first case, can choose to either do nothing or revoke in the second case, and must actively maintain and broadcast that revocation list for the third case. When someone says "hey I accidentally clobbered that private key can I please have a new cert for this new key," you generally don't add the old cert to the revocation list because why would you.