9 ms·
Red Hat confirms security incident after hackers breach GitLab instance
- INTPenis 1y agoRed Hat, I am very disappointed. We're all ISO27001 everywhere, separation of data, and separation of network resources. But you keep our data in your github repo?
- ExoticPearTree 1y agoData is separated in different repositories, per the story. And, never forget: what a company preaches and advertises is not the same with what the company is actually doing.
- zingababba 1y agoYes, I've been in fortune 100 IT security for awhile now. When it comes to passing audits its a shit ton of misdirection. Also, here is some more information about this breach: https://x.com/intcyberdigest/status/1973422846396473765 https://x.com/intcyberdigest/status/1973422846396473765
- everdrive 1y agoEvery time I've been involved in audits at a company, my boss will tell me "let me tell you how to talk to auditors," which ends up meaning lie by omission, imply that things are in good standing without making strictly false statements, and otherwise just make the auditors go away. It all seems silly, but maybe it should be thought of like the court system? An adversarial process whereby each side is vying for its own interests?
- Spooky23 1y agoThat’s a good point of view. 90% of the time, they are checking boxes. But if they are fishing, you have to be careful because they generally are bad at understanding anything, but good at manipulating the audit rules to frame things in such a way so they can “catch a big fish”.
- behnamoh 1y agoif you find it unethical, can't you leave anonymous tips for the auditers?
- delusional 1y agoSure, if you don't want a job.
- deleted 1y ago[deleted]
- ExoticPearTree 1y agoWith auditors you talk like you would either talk on a deposition or on the witness stand: do not say more than what you were asked, do not make assumptions, do not try to be helpful in any way, do not offer more data than asked for. Is it really OK? Not necessarily, but on the other hand you don't want to spend the rest of your life answering even more questions from other people the auditors might bring in to help them understand your helpful explanations. I learned this the hard way, assuming auditors are logical and understand technology.
- dijit 1y agoThe issue is, it’s very easy to understand what’s not being said for a reasonably intelligent person. A person who is used to interviewing people will be able to tell right away.
- unethical_ban 1y agoThat may sound bad or immoral by the company, but know that auditors have the own ambition and mo ey to think about, and will try to mark any possible thing as a serious problem regardless of whether it is. Yes, it is highly adversarial and the best compromise I've seen is to have an internal audit team that is separate organizationally from IT, but has to withstand peer review if they claim anything is a real problem.
- baobun 1y agos/should/could/ Your boss is bad apple and so are you if you adopt their ways.
- jandrusk 1y agoVPN profiles? Best start revoking/renewing certs and keys. Geez, why would you store VPN profiles there?
- mmh0000 1y agoYou've got to look at ISO27001 from the perspective of the Sales Rep, not from an Engineer. In theory, being ISO27001 means that you're environment follows best practices and has a somewhat sane security posture. To the business people, a new customer demands that you have ISO27001 certification before they'll sign the $$$$ contract. The salesperson does not care HOW you get the certificate, just that you have it, they need this contract signed! The department wasn't designed with security in mind, so implementing everything required by ISO will take many months. But sales needs $$$$ now! The CEO, CFO, and CTO are aligned: money now! So, there's high pressure to pass the audit quickly. You implement what you can, you weasle your way around the things that will take too long. Those things are "out of scope" or "testing databases". You implement MFA while the auditor is auditing, but you know it breaks developers' workflows and there isn't a quick fix, so you turn MFA back off once the audit is complete.... TA-DA! We're ISO27001 certified! But we're no more secure than we were before.
- throwaway127482 1y agoEngineers who are smart enough / talented enough, and who feel secure, can push back on security issues even if it will hold up a deal. This tells me that the most valuable engineers at Red Hat either do not push back enough on security concerns, or don't care enough (or aren't experienced enough) to know that the concerns exist in the first place, or they feel insecure in their position.
- array_key_first 1y agoUltimately, devs can't get sales reps fired, but sales reps can absolutely get devs fired. Depending on how dysfunctional the org is, there's no super dev anywhere who can fix it. You just shut up, do bad things knowing theyre bad, or get fired.
- anthk 1y agoIt should be the opposite. For every big engineering issue happened because of the sales' dept pressures, the sales reps would have their asses out of any company.
- baobun 1y ago> Correction: After publishing, Red Hat confirmed that it was a breach of one of its GitLab instances, and not GitHub. Title and story updated. > After publishing our story, Red Hat confirmed that the security incident was a breach of its GitLab instance used solely for Red Hat Consulting on consulting engagements, and not GitHub. > While Red Hat did not respond to any further questions about the breach, the hackers told BleepingComputer that the intrusion occurred approximately two weeks ago.
- 6c696e7578 1y agoGitLab, not GitHub. I think the distinction is that you can have a on-prem GitLab (as well as hosted online). The implication here being that RedHat probably had very relaxed account security.
- elicash 1y ago"The hackers stated that they attempted to contact Red Hat with an extortion demand but received no response other than a templated reply instructing them to submit a vulnerability report to their security team." Just hilarious
- 1970-01-01 1y agoYou didn't give the kicker: "According to them, the created ticket was repeatedly assigned to additional people, including Red Hat's legal and security staff members." Summarized: Given enough eyeballs, all extortion demands are fallow.
- nomilk 1y agofallow == marked by inactivity Thanks, hadn't encountered this word before.
- elicash 1y agoAnd then there's more, via 404: “Since RedHat doesn't want to answer to us,” the hackers wrote in a channel on Telegram viewed by 404 Media, suggesting they have attempted to contact Red Hat. [...] “We have given them too much time already to answer lol instead of just starting a discussion they kept ignoring the emails,” the message added. In another message, the group said it had “gained access to some of their clients' infrastructure as well, already warned them but yeah they preferred ignoring us.” https://www.404media.co/red-hat-investigating-breach-impacting-as-many-as-28-000-customers-including-the-navy-and-congress/ https://www.404media.co/red-hat-investigating-breach-impacti...
- zb3 1y agoTelegram name: "thecrimsoncollective"
- gryfft 1y ago[flagged]
- Kaytaro 1y agoNo this was all Red Hat. IBM would never let consultants use something convenient like GitHub, they would be forced to use some crappy internal webapp powered by watson.
- bonzini 1y agoIBM uses GitHub Enterprise.
- RickJWagner 1y agoRetired Red Hatter here. I wouldn’t be quick to blame IBM. Red Hat and IBM both take security very seriously, and regard it as central to the mission. IBM also has deep enough pockets to devote serious resources to whatever they put their mind to. Security is just hard. Procedures can be written, but people make mistakes, forget rules, etc. The procedures also have to be constantly updated to keep up with new and innovative attacks. It’s a never ending battle. Sorry to see this happen to Red Hat. I am confident that right now all hands are on deck working on remediation.
- loudmax 1y agoSecurity is hard, but running a large organization is also hard. The danger is that the entire organization becomes overrun by managers whose guiding principle is CYA, and then things slowly grind to a halt. Eventually you wind up with security mandates that have no relation to what's actually deployed because the people nominally responsible for security are evaluating metrics that are years out of date.
- 1oooqooq 1y agoi doubt old red hat wouldn't have warned clients right away.
- deleted 1y ago[deleted]
- dang 1y ago[stub for offtopicness] (title fixed now)
- jmclnx 1y ago[flagged]
- jasonjmcghee 1y agoSounds like they got their hands on authentication tokens. Through RedHat, not GitHub.
- tzs 1y agoIt looks to me like the article is talking about authentication tokens that they found in the data they took from the breach.
- reactordev 1y ago>”They allegedly found authentication tokens, full database URIs, and other private information in Red Hat code and CERs, which they claimed to use to gain access to downstream customer infrastructure.” They published CERs somewhere that had access keys and urls. Probably to somewhere that it wasn’t authorized to be published or shared and they got a hold of it. Using that, they got a hold of everything else. More CERs, GitHub repos using access tokens, vpn credentials to all the hottest players. At this point, you’d have to tear down and rebuild to undo the damage. Rotating certs, keys, IP’s, the whole nine yards.
- stingrae 1y agoCorrection Posted: "After publishing, Red Hat confirmed that it was a GitLab account breach, not GitHub."
- erikerikson 1y ago"Correction: After publishing, Red Hat confirmed that it was a breach of one of its GitLab instances, and not GitHub. Title and story updated." Title needs updating
- temphaaa 1y ago[dead]
- hoffie 1y agoThere seems to be an official statement by Red Hat now: https://access.redhat.com/articles/7132207 https://access.redhat.com/articles/7132207
- commandersaki 1y agoOh that's surprising, it isn't behind a paywall.
- QuantumNoodle 1y agoBefore even reading the article, the title made me lol as I thought the open sourced code was stolen XD