7 ms·
Why am I hearing about this on checks the fucking date on May 5th instead of, like, month ago.
by oxff 4y ago
Why am I hearing about this on checks the fucking date on May 5th instead of, like, month ago.
- VWWHFSfQ 4y agoMy understanding is they didn't even know about this until GitHub told them on April 13th. I'm guessing something got triggered in GitHub's system by a flurry of tokens issued to Heroku trying to enumerate private repositories. If the attacker had just played it low and slow they might never even have known at all. Who knows how long Heroku's internal systems were compromised.
- cmeacham98 4y agoEven if that's the case, it's still way after April 13th.
- chii 4y agoit might be that it took this amount of time to establish the facts of the events. If they recounted an incorrect version early, it might do more damage than not telling it. I dont know if the github disclosure "includes" heroku's disclosure : https://github.blog/2022-04-15-security-alert-stolen-oauth-user-tokens/ https://github.blog/2022-04-15-security-alert-stolen-oauth-u... - but it was at least april 15th - close-ish to when the event occurred.
- bigDinosaur 4y agoIs it ever true that earlier indications that credentials should be rotated leads to worse outcomes, though, as just one example? I'm sure I've received emails of the form: we suspect there may have been a breach, so we're forcing password resets, and have always taken that fine.
- dev_tty01 4y agoHeroku reported it on 4/15. Read the beginning of the string of updates on the notification page posted here. Also, https://news.ycombinator.com/item?id=31048646 https://news.ycombinator.com/item?id=31048646
- heartbreak 4y agoAnd they reported that the credentials were leaked on 5/3. That took a long time.
- dev_tty01 4y agoWhich credentials are you referring to? They reported the loss of OAuth tokens on April 15. What am I missing? https://status.heroku.com/incidents/2413 https://status.heroku.com/incidents/2413 "On April 13, 2022, Salesforce Security was notified by GitHub that a subset of Heroku’s GitHub private repositories, including some source code, was downloaded by a threat actor on April 9, 2022. Based on Salesforce’s initial investigation, it appears that unauthorized access to Heroku's GitHub account was the result of a compromised OAuth token. Salesforce immediately disabled the compromised user’s OAuth tokens and disabled the compromised user’s GitHub account. Additionally, GitHub reported that the threat actor was enumerating GitHub customer accounts using OAuth tokens issued to Heroku’s OAuth integration dashboard hosted on GitHub. Based on the information GitHub shared with us, we are investigating how the threat actor gained access to customer OAuth tokens. The compromised tokens could provide the threat actor access to customer GitHub repos, but not customer Heroku accounts. With the access to customer OAuth tokens, the threat actor may have read and write access to customer GitHub repositories connected to Heroku. Given the incident is still active, please review the recommended actions provided below." Posted 21 days ago, APR 15, 2022 23:36 UTC
- heartbreak 4y agoYou’re missing the 5/3 update about username and password credentials. > our investigation also revealed that the same compromised token was leveraged to gain access to a database and exfiltrate the hashed and salted passwords for customers’ user accounts. From the link we are commenting on.
- dev_tty01 4y agoGot it. Thanks.
- latortuga 4y agoThis has been an ongoing security incident communicated through multiple channels since the day that github announced it. I've got a dozen emails or more in my inbox, the heroku dashboard includes mention of it, and the status page includes information about it.
- jweir 4y agoI first received word about this on 4/15 via an email from Salesforce.
- deleted 4y ago[deleted]
- nightpool 4y agoSalesforce informed you that their entire DB was compromised, and not just the Github OAuth tokens, as they've been saying for weeks? The first indication that anything except a Github specific DB was compromised was Tuesday, when they started telling people that seemingly-all non-DB non-addon credentials were going to roll.
- stevebmark 4y agoBecause SalesForce owns Heroku, and SalesForce has notoriously poor incident handling.
- monkeymonkey 4y agohttps://www.heavybit.com/library/video/every-minute-counts-coordinating-herokus-incident-response/ https://www.heavybit.com/library/video/every-minute-counts-c...
- bgentry 4y agoHi, speaker in that linked video here :) A lot has changed at Heroku in the past 8 years since I left, particularly in the direction of being subsumed into the greater Salesforce org. My working assumption is that everything left there is being done "the Salesforce way" at this point. Take that to mean what you will, but it seems pretty clear we're long past the days of openly communicating with customers as quickly as you have relevant/important information to share with them.