5 ms·
you forgot "that would have resulted in the disclosure of customer credentials". I'm speculating, but having been a fly on the wall at this kind of meeting bef
by drewcrawford 11y ago
you forgot "that would have resulted in the disclosure of customer credentials".
I'm speculating, but having been a fly on the wall at this kind of meeting before, here's my theory of how this went down:
Manager: "So somebody got the key to generate one-time password tokens for PagerDuty. How did that happen?"
Engineer: "I have no idea."
Manager: "What about that Lish vulnerability? Could it have been that?"
Engineer: "Could have been."
Manager: "We don't know?"
Engineer: "I mean, they could have gotten it that way, but we don't have logging for security vulnerabilities, because, you know, we would just fix them instead of logging that they occurred..."
Manager: "Okay, I want a second opinion. Time to bring in $SECURITY_PARTNER."
$SECURITY_PARTNER: "What Engineer said. And here's a bill for our time."
Manager: "What about if the customer screwed up and lost a device? Could it be that?"
Engineer: "It could be that."
Support: "PagerDuty said something about wiping their phone."
Engineer: "If they did lose a device, it would be indistinguishable from the evidence we have."
Manager: "So, maybe we weren't hacked after all?"
Engineer: "Maybe not."
Manager: "Any other theories?"
Engineer: "I just want to go on record... AGAIN... that I don't trust our ColdFusion infrastructure at all. The people responsible have all left, and none of us understand it. We need to rewrite in a cooler language, like Python. And hire some actual security people."
Manager: "Sigh. We've gotten enough flak over this that I guess I'll put it in the budget. Okay, good meeting everyone. I'll write this up in a blogpost, because we need to show transparency. That way customers are at least as confused as we are."
- TheSwordsman 11y agoIn our case someone logged in to the Linode Manager first. Whomever logged in was confident they knew the password of the user and the MFA token. That would indicate that Linode's password hashes may have been cracked offline.
- mikegrb 11y agoHah, pretty close except the only person responsible is still there.