6 ms·
GitHub Co pilot was doing this earlier as well. I am not talking about giving your token to Claude or gpt or GH co pilot. It has been reading private repos si
by kapitanjakc 1y ago
GitHub Co pilot was doing this earlier as well.
I am not talking about giving your token to Claude or gpt or GH co pilot.
It has been reading private repos since a while now.
The reason I know about this is from a project we received to create a LMS.
I usually go for Open edX. As that's my expertise. The ask was to create a very specific XBlock. Consider XBlocks as plugins.
Now your Openedx code is usually public, but XBlocks that are created for clients specifically can be private.
The ask was similar to what I did earlier integration of a third party content provider (mind you that the content is also in a very specific format).
I know that no one else in the whole world did this because when I did it originally I looked for it. And all I found were content provider marketing material. Nothing else.
So I built it from scratch, put the code on client's private repos and that was it.
Until recently the new client asked for similar integration, as I have already done that sort of thing I was happy to do it.
They said they already have the core part ready and want help on finishing it.
I was happy and curious, happy that someone else did the process and curious about their approach.
They mentioned it was done by their in house team interns. I was shocked, I am no genius myself but this was not something that a junior engineer let alone an intern could do.
So I asked for access to code and I was shocked again. This was same code that I wrote earlier with the comments intact. Variable spellings were changed but rest of it was the same.
- RedCardRef 1y agoWhich provider is immune to this? Gitlab? Bitbucket? Or is it better to self host?
- Aurornis 1y agoGitHub won’t use private repos for training data. You’d have to believe that they were lying about their policies and coordinating a lot of engineers into a conspiracy where not a single one of them would whistleblow about it. Copilot won’t send your data down a path that incorporates it into training data. Not unless you do something like Bring Your Own Key and then point it at one of the “free” public APIs that are only free because they use your inputs as training data. (EDIT: Or if you explicitly opt-in to the option to include your data in their training set, as pointed out below, though this shouldn’t be surprising) It’s somewhere between myth and conspiracy theory that using Copilot, Claude, ChatGPT, etc. subscriptions will take your data and put it into their training set.
- suddenlybananas 1y agoCompanies lie all the time, I don't know why you have such faith in them
- Aurornis 1y agoAnonymous Internet comment section stories are confused and/or lie a lot, too. I’m not sure why you have so much faith in them. Also, this conspiracy requires coordination across two separate companies (GitHub for the repos and the LLM providers requesting private repos to integrate into training data). It would involve thousands or tens of thousands of engineers to execute. All of them would have to keep the conspiracy quiet. It would also permanently taint their frontier models, opening them up to millions of lawsuits (across all GitHub users) and making them untouchable in the future, guaranteeing their demise as soon a single person involved decided to leak the fact that it was happening. I know some people will never trust any corporation for anything and assume the worst, but this is the type of conspiracy that requires a lot of people from multiple companies to implement and keep quiet. It also has very low payoff for company-destroying levels of risk. So if you don’t trust any companies (or you make decisions based on vague HN anecdotes claiming conspiracy theories) then I guess the only acceptable provider is to self-host on your own hardware.
- suddenlybananas 1y agoI really don't see how tens of thousands of engineers would be required.
- brian-armstrong 1y agoWith the current admin I don't think they really have any legal exposure here. If they ever do get caught, it's easy enough to just issue some flimsy excuse about ACLs being "accidentally" omitted and then maybe they stop doing it for a little while. This is going to be the same disruption as Airbnb or Uber. Move fast and break things. Why would you expect otherwise?
- Covenant0028 1y ago
- digi59404 1y agoSelf hosted GitLab with a self-hosted LLM Provider connected to GitLab powering GitLab Duo. This should ensure that the data never gets outside your network, is never used in training data, and still allows you/staff to utilize LLMs. If you don’t want to self host an LLM, you could use something like Amazon Q, but then you’re trusting Amazon to do right by you. https://docs.gitlab.com/administration/gitlab_duo_self_hosted/ https://docs.gitlab.com/administration/gitlab_duo_self_hoste...
- Aurornis 1y agoIf you found your exact code in another client’s hands then it’s almost certainly because it was shared between them by a person. (EDIT: Or if you’re claiming you used Copilot to generate a section of code for you, it shouldn’t be surprising when another team asking Copilot to solve the same problem gets similar output) For your story to be true, it would require your GitHub Copilot LLM provider to use your code as training data. That’s technically possible if you went out of your way to use a Bring Your Own Key API, then used a “free” public API that was free because it used prompts as training data, then you used GitHub Copilot on that exact code, then that underlying public API data was used in a new training cycle, then your other client happened to choose that exact same LLM for their code. On top of that, getting verbatim identical output based on a single training fragment is extremely hard, let alone enough times to verbatim duplicate large sections of code with comment idiosyncrasies intact. Standard GitHub Copilot or paid LLMs don’t even have a path where user data is incorporated into the training set. You have to go out of your way to use a “free” public API which is only free to collect training data. It’s a common misconception that merely using Claude or ChatGPT subscriptions will incorporate your prompts into the training data set, but companies have been very careful not to do this. I know many will doubt it and believe the companies are doing it anyway, but that would be a massive scandal in itself (which you’d have to believe nobody has whistleblown)
- pinoy420 1y ago[dead]
- cmiles74 1y agoI believe the issue here is with tooling provided to the LLM. It looks like GitHub is providing tools to the LLM that give it the ability to search GitHub repositories. I wouldn't be shocked if this was a bug in some crappy MCP implementation someone whipped up under some serious time pressure. I don't want to let Microsoft of the hook on this but is this really that surprising? Update: found the company's blog post on this issue. https://invariantlabs.ai/blog/mcp-github-vulnerability https://invariantlabs.ai/blog/mcp-github-vulnerability
- throwaway314155 1y ago
- 1oooqooq 1y agothinking a non enterprise GH repo to be out of reach from Microsoft is like giving your phone for Facebook authentication and thinking they won't add it to their social graph matching.
- ikiris 1y agoYou're completely leaving out the possibility that the client gave others the code.
- kapitanjakc 1y agoLol, never thought about that, it's highly possible.
- 6Az4Mj4D 1y agoIn GitHub Co pilot if we say dont use my code option for training does this still leaks your private code?
- ZYbCRq22HbJ2y7 1y agoRead the privacy policy and terms of use https://docs.github.com/en/site-policy/privacy-policies/github-general-privacy-statement#from-you https://docs.github.com/en/site-policy/privacy-policies/gith... IMO, You'd have to be naive to think Microsoft makes GitHub basically free for vibes.
- josteink 1y agoGithub copilot is most definitely not free for Github enterprise customers.
- deleted 1y ago[deleted]
- ZYbCRq22HbJ2y7 1y agoI didn't realize we were talking about that.
- Shekelphile 1y agoYes. Opt-outs like that are almost never actually respected in practice. And as the OP shows, microsoft is intentionally giving away private repo access to outside actors for the purpose of training LLMs.
- alfiedotwtf 1y ago“With comments intact” … SCO Unix Lawyers have entered the chat
- ZYbCRq22HbJ2y7 1y ago> I know that no one else in the whole world did this because when I did it originally I looked for it. Not convincing, but plausible. Not many things that humans do are unique, even when humans are certain that they are. Humans who are certain that things that they themselves do are unique, are likely overlooking that prior.
- bastardoperator 1y agoAgreed, ask it for the cutoff date. I did, June 2024...
- yellow_lead 1y agoIt seems you're implying Github Copilot trained on your private repo. That's a completely separate concern than the one raised in this post.