14 ms·
Repositories held for ransom by using valid credentials
- nicolaslem 7y agoThe title on HN is clickbait, the article mentions Gitlab users storing their own Gitlab password/tokens insecurely. It doesn't look like "Gitlab was compromised" to me. The original title is "Critical security announcement: Suspicious git activity detected".
- sytse 7y agoCorrect, users across GitLab and GitHub as been affected and in all cases valid credentials were used. Also see https://www.bleepingcomputer.com/news/security/attackers-wiping-github-and-gitlab-repos-leave-ransom-notes/ https://www.bleepingcomputer.com/news/security/attackers-wip... We are updating our title to better reflect what happened.
- fillskills 7y agoSorry but the title still doesn't reflect the actual issue which per your link is : "Attackers Wiping GitHub and GitLab Repos, Leave Ransom Notes"
- sytse 7y agoDid you see our new title? https://gitlab.com/gitlab-com/www-gitlab-com/commit/447f2330391992a16c54427d2252b5e3995d207b https://gitlab.com/gitlab-com/www-gitlab-com/commit/447f2330...
- paxys 7y agoGitlab was NOT compromised. Someone found passwords/tokens for Gitlab repositories exposed on the internet and held them for ransom.
- mortehu 7y agoPurging user data is one of the most common action attackers take when compromising an account. This makes it prudent for storage service providers to silently delay mass deletions to the extent allowed by their data deletion policy/GDPR to allow time to discover any breaches, or perhaps require second factor verification like a link sent via email.
- edwinbalani 7y ago"Gitlab.com was compromised" is a bad title. _Accounts on_ GitLab, and the credentials to access them, were compromised, but the title suggests that the whole platform was affected, which doesn't seem to be the case.
- jjeaff 7y agoLooks like they changed the title. But I would have to say that the ability to delete a full repo with the credentials is a bit of a vulnerability. To me, it seems like a good measure would be to mark deleted repos as "delete requested" then notify the users involved and give them a week or two to undo a total delete. Especially if it is an older repo with lots of commits.
- rmkrmk 7y ago> We believe that no data has been lost, unless the owner/maintainer of the repository did not have a local copy and the GitLab copy was the only one. Too bad they don't make backups of users repositories?
- phoe-krk 7y agoGitLab is not a repository backup service. They are only required to hold multiple copies of what is the current version of the repository for their own hard drive fault tolerance purposes; they are not required to hold copies of the repository from a day, week, month, or year ago.
- rmkrmk 7y agoWas just a question, don't get me wrong here. Thought they would do this for their managed service.
- zoidb 7y agoYou can see how GitLab handles repository backups on the following page https://about.gitlab.com/handbook/engineering/infrastructure/design/disaster-recovery/#repository-data https://about.gitlab.com/handbook/engineering/infrastructure...
- the_duke 7y agoThey do infact create snapshots twice a day and keep them for two weeks. https://about.gitlab.com/handbook/engineering/infrastructure/design/disaster-recovery/#repository-data https://about.gitlab.com/handbook/engineering/infrastructure...
- vasco 7y agoThe current title "Gitlab.com Was Compromised" doesn't seem accurate. There's someone (or a group) currently attacking online repositories (gitlab is not the only affected provider) using passwords found in scans for files like .gitconfig's and the like. Unless new information comes to light about gitlab specifically being compromised, I'd say this is more about individual private repos being on the sights of a targetted attack.
- commandersaki 7y agoSo git doesn't let you add the `.git` to the index. Most reports I've seen mention that SourceTree was used as a git client. Is it possible that SourceTree committed .git and pushed it to remotes which were then scraped?
- plttn 7y agohttps://twitter.com/bad_packets/status/1124429828680085504 https://twitter.com/bad_packets/status/1124429828680085504 Looks like someone was scraping for `.git/config`
- sylens 7y agoA bit of a misleading headline. Somebody had passwords/tokens for certain repos, either from previous breaches at other services or those passwords being stored in plaintext as part of a deployment.
- fmhul 7y ago>The breaches seem to rely on the attacker having knowledge of the affected users passwords in order to wipe their Git repositories and hold them for ransom. Yeah, until I go to my computer and use "git push" again. No? Also gitsbackup.com is registered but has no A/MX records so...
- tyingq 7y ago"Also gitsbackup.com is registered but has no A/MX records so..." Gitlab should really note that in their blog posts and emails to users. Just in case someone is thinking of paying the ransom.
- sytse 7y agoWe agree that paying a ransom doesn't guarantee any further actions on the part of the attackers. But in our blog post we want to stick to what we know and can influence and not talk about an external DNS record that can be added at any time.
- nerdy 7y agoBased upon other comments the title has already been changed several times. It still suggests Gitlab's infrastructure (internally) was compromised: "Suspicious git activity detected on Gitlab" Something like "Gitlab users' repos held for ransom" seems more appropriate.
- deleted 7y ago[deleted]
- kzrdude 7y agoSince the threat is to make the code public, there is nothing more gitlab can do to shut down the attempted blackmail. It seems unlikely to be a real threat to most?
- tyingq 7y agoOdd threat in that paying the ransom doesn't assure they wouldn't make it public anyway. Also, someone else noted the ransom email domain has no MX or A records, so the instructions to email them won't work. They seem to be hoping someone will blindly pay the ransom.
- 1023bytes 7y agoSome other info: https://www.theregister.co.uk/2019/05/03/git_ransomware_bitcoin/ https://www.theregister.co.uk/2019/05/03/git_ransomware_bitc...
- yoz-y 7y agoOut of all things to hold for ransom git repos seem like a bad idea. Most of the time there are multiple clones lying around anyways. I agree that having the source code leak can be bad news, but the code itself being secret should not be a critical part of the business.
- Operyl 7y agoSo you’re telling me that if, say, Google’s Search algorithm got leaked it wouldn’t seriously hinder their business? There’s definitely cases where leaking a business’ code can be pretty disastrous.
- raverbashing 7y ago> So you’re telling me that if, say, Google’s Search algorithm got leaked Google's search algorithm was public for a long time [0], it's been improved now and more under the curtains, but the answer is no. Google wins not only by technology, but also by size of the index [0] https://en.wikipedia.org/wiki/PageRank https://en.wikipedia.org/wiki/PageRank
- tyingq 7y agoI believe Google has also bolted enough ML to search that they couldn't really recreate current search with just code.
- rightbyte 7y agoAlso the code base is probably so big and esoteric that there is no way to replicate the services without knowledge external to the repo. You can get bits and parts and it would be disastrous for Google, but you would need 1000 engineers to reverse engineer the source code into some working search engine or what ever service you would want to replicate. I've been thinking about what if the competitors got hold of my employers code base. I think it would require years of reverse engineering to get anything useful out of it for their product, and that they could just as well put that effort into their product directly without espionage.
- avar 7y ago> We believe that no data has been lost, unless the [...] GitLab copy was the only one. One difference between how GitLab and GitHub run their infrastructure is that GitLab doesn't keep reflogs, and uses git's default "gc" settings. As a result they won't have the data in question anymore in many cases[1]. Well, I don't 100% know that for sure, but it's the default configuration of their software, and I'm assuming they use like that themselves. Whereas GitHub does keep reflogs, and runs "git repack" with the "--keep-unreachable" option. They don't usually delete git data unless someone bothers to manually do it, and they usually have data to reconstruct repositories as they were at any given point in time. GitHub doesn't expose that to users in any way, although perhaps they'd take pity on some of their users after such an incident. This isn't a critique of GitLab, just trivia about the storage trade-offs different major Git hosting sites have made, which might be informative to some other people. I'm surprised no major Git hosting site has opted to provide such a "we have a snapshot of every version ever" feature. People would probably pay for it, you could even make them opt to pay for access to backups you kept already if they screwed things up :) 1. Well, maybe as disaster backups or something. But those are harder to access...
- lostmyoldone 7y agoRe GitHub keeping unreachable data, if I understand it right, isn't that GitHub painting a giant target on their back? Wouldn't that that imply every secret accidentally committed and then 'deleted' is still accessible, when one would expect it not to be? It's one thing to have your source code in the wild, but pairing it up with thought-to-be-deleted secrets would be an absolute disaster. Certainly one should not ever keep using a secret once it has escaped into a Git repo, but I'm sure it happens quite frequently.
- hnarn 7y ago> Wouldn't that that imply every secret accidentally committed and then 'deleted' is still accessible This should be a moot point because anyone (in IT) should realize that an accidentally committed secret is now 100% public for all eternity and needs to be rendered irrelevant to restore secure operations.
- chiefalchemist 7y ago- Sidebar For those who might not be aware: It's possible to configure your .git config to push to different remotes.
- kissgyorgy 7y agoThis is why I'm hosting GitLab myself even if I'm the sole user of the instance. For one thing I'm less of a target, the other is that this is not the first major problem with the hosted GitLab. I have better uptime for my instance than the hosted version.
- blumomo 7y agoAlso GitHub users are affected. By the time of writing 379 public GitHub repos have been compromised: https://github.com/search?o=desc&q=1ES14c7qLb5CYhLMUekctxLgc1FV2Ti9DA&s=indexed&type=Code https://github.com/search?o=desc&q=1ES14c7qLb5CYhLMUekctxLgc...
- cdubzzz 7y agoI like this threat for open source software published on GH -- If we dont receive your payment in the next 10 Days, we will make your code public or use them otherwise.
- akoster 7y ago(At least for public repos) Isn’t this almost a compliment? (Having another person host your code and use it?) xD
- tyingq 7y agoThe "Global Association of Risk Professionals" got hit. That should be a fun meeting. https://github.com/GARPDev https://github.com/GARPDev
- Ayesh 7y agoThere was a Docker Hub breach a few days ago, that's probably related. I took a good look at how my personal tokens were used in Github and Gitlab. - Enable 2FA. - Enable Commit signing with GPG. for the past 2-3 years, I have slowly moved to sign commits and tags. GPG keys take a log of hygiene to work with (sub keys, revocation, etc), but they definitely can help in a situation like. Git is a distributed VCS. If you have a repo cloned in a secure location (your server, Dev machine, etc), that is just as good as your Gitlab/hub hosted copy.
- jcims 7y agoThe ‘play with docker’ site used to make it pretty easy to see what others were up to and snag git creds if they left them around.
- agildehaus 7y agoMandated 2FA should really be a thing, especially on tech-oriented sites with such importance.
- jaden 7y agoI agree, but read that at least one of the users had 2FA enabled and still lost their repos. They said they received no notification emails either.
- smudgymcscmudge 7y agoBoth GitLab and GitHub allow organizations to require members to use 2fa. So the option is there.
- ssnistfajen 7y agoSomeone I know had one of their private repos on GitHub replaced despite having 2FA enabled so it may have been from a leaked personal access token somewhere. What's odd was that this user has push access to multiple active private repos yet only one was hit with the ransom.
- sara7262 7y agoFriendship between cat and dog see in video how much they are teasing each others also they are loving https://funlandtube.wordpress.com/2019/05/04/friendship-between-cat-and-dog/ https://funlandtube.wordpress.com/2019/05/04/friendship-betw...
- localhostdotdev 7y agokinda hoping "testing weak passwords on your existing user's passwords" becomes standard practice at some point.
- tyingq 7y agoGitHub does that...it warns you with a banner.
- a3_nm 7y agoI don't get the ransom thing: users of a git repository have a clone of the repo that contains the whole history, no? So isn't it trivial to recreate the repository?
- billconan 7y agoI do not have local clones of my old projects.
- facorreia 7y agoNo, replication is not backup.
- ph4 7y agoThe attacker is also threatening to make these private repos public, or misuse their access to the repos in other ways (likely additional types of breaches).
- duxup 7y agoFor some repositories the code may be of little concern if the hacker shares it with the world or deletes it.... but it could impact some users.
- 2T1Qka0rEiPr 7y agoWeird aside question: I notice the article says "at approximately 10:00pm GMT". Can someone explain why GMT might be chosen as a reference point here? Is there something I'm missing about the usage of GMT (and not UTC). It just seems particularly odd given that GMT is not (to my knowledge) actually being used as a concrete time-zone at the minute (BST is in effect for daylight savings).
- s_ngularity 7y agoIn the UK, GMT is often used to refer to "the current British time" both GMT/BST. I've seen the same in the US where people say EST but mean EDT.
- 2T1Qka0rEiPr 7y agoIs it? I mean, I'm British and I'm not aware of this.
- stordoff 7y agoI've seen a few people do it, but it's also usually corrected as been wrong.
- rahimnathwani 7y agoWhenever someone says GMT/PST/EST during the summer, I need to ask them to confirm whether they mean BST/PDT/EDT. Whatever I assume, I might be off by an hour.
- greggyb 7y agoI usually attribute it to mild ignorance, not in a bad way. For a long time GMT was a good reference point. Times have changed. I used to work with a gentleman who would always schedule meetings on the phone as: > Great, let's put that on the schedule for 2:00 o'clock Eastern Standard Time. There was always a bit of officiousness to his tone and I think he just liked the idea of being precise. And he certainly was precise. He was also off by an hour for half the year. Somehow no one ever missed a meeting, though. I always sat on the other side of the room and ground my teeth.
- exabrial 7y agoU2F token, which is supported by gitlab, would have stopped this
- _bxg1 7y ago"...to wipe their Git repositories and hold them for ransom." What an idiotic strategy to take with git repositories. Every local copy is a complete and fully-functioning copy of not just the code, but all history, etc. It's a non-centralized protocol.
- based2 7y agohttps://security.stackexchange.com/questions/209448/gitlab-account-hacked-and-repo-wiped https://security.stackexchange.com/questions/209448/gitlab-a...
- theWheez 7y agoA client of mine was hit by this. Makes me think I aught to go into security.. I brought up many concerns, and specifically about git access (using very weak mechanisms), a few months ago, to which I was told to "clean things up when I can" but that it wasn't a priority.
- snowwolf 7y agoWhile the fault lies with the users for not following security best practices, including enabling 2FA there are things gitlab/any site can do to help defend against these sorts of attacks. Some suggestions: Treat logins from datacenters as suspicious. (In this case the IP block identified belongs to World Hosting Farm Limited). Treat logins from a new/different ISP as suspicious. Limit access to the account and verify the login via email. It’s not foolproof but as part of a defense in depth strategy it can be quite effective.
- gitlab-security 7y agoThank you for your feedback and suggestions. Unfortunately, for each of these proposals, we're likely to have users asking us why we are restricting and/or blocking access. A better defense-in-depth strategy would be to scan each public repo for credentials, and act accordingly when credentials are discovered in repos. We are working on this strategy, currently.
- snowwolf 7y agoThat doesn’t help stop the attacks using breach lists that are even more prevalent. You could start with email warnings of suspicious activity and fine tune the model parameters based on feedback from false positives. But generally a login from a device that has no previous cookie, from an ASN the account has never used before, especially if that ASN is a known data center, that then immediately attempts a destructive action, should be a pretty big warning flag.