4 ms·

Github wasn't abused to train LLMs. Whatever website existed that openly hosted OSS repositories would have been scrapped if it had such a large number of repos

Github wasn't abused to train LLMs. Whatever website existed that openly hosted OSS repositories would have been scrapped if it had such a large number of repos. You don't need to be Microsoft to train on github.

johannes1234321 3y ago
The question is: How much private repos went into copilot? I guess for specific software (how to use a commercial database? How to use some enterprise framework?) they'd find more relevant samples there, which might be worthwhile scanning.
- 542458 3y ago
  GitHub has always said that they only train on public repos. I don’t see why they’d lie about that - including private repos would only add a relatively small amount of data to the corpus and would risk alienating all their corporate customers. > GitHub Copilot is trained on all languages that appear in public repositories https://docs.github.com/en/copilot/overview-of-github-copilot/about-github-copilot-for-individuals#about-github-copilot https://docs.github.com/en/copilot/overview-of-github-copilo...
  - sharminglkw 3y ago
    When copilot was originally released, I was able to get it to generate verbatim my extremely niche and poorly written code from a completely private repo. It was identifiable by the bad variable names that I used. I cant say they traines it on all pricate repos, but they definitely trained it on my private repos.
    - 542458 3y ago
      Did you reach out to any tech press figures or publicize this in any way? It would have been a fairly important story and should have been pretty easy for somebody else to reproduce by generating the same completions and comparing them to your commit history. I don’t mean to be dismissive but unless you did something like that or documented it in some way it’s hard to give too much weight to a “source: trust me” comment from a brand-new HN account.
      - sharminglkw 3y ago
        I actually posted it on HN! If you look hard enough you might even find it. Just don't dox me, thanks.
    - IshKebab 3y ago
      Bullshit. That would be global news.
      - sharminglkw 3y ago
        [flagged]

johannes1234321 3y ago

The question is: How much private repos went into copilot? I guess for specific software (how to use a commercial database? How to use some enterprise framework?) they'd find more relevant samples there, which might be worthwhile scanning.

542458 3y ago
GitHub has always said that they only train on public repos. I don’t see why they’d lie about that - including private repos would only add a relatively small amount of data to the corpus and would risk alienating all their corporate customers. > GitHub Copilot is trained on all languages that appear in public repositories https://docs.github.com/en/copilot/overview-of-github-copilot/about-github-copilot-for-individuals#about-github-copilot https://docs.github.com/en/copilot/overview-of-github-copilo...
- sharminglkw 3y ago
  When copilot was originally released, I was able to get it to generate verbatim my extremely niche and poorly written code from a completely private repo. It was identifiable by the bad variable names that I used. I cant say they traines it on all pricate repos, but they definitely trained it on my private repos.
  - 542458 3y ago
    Did you reach out to any tech press figures or publicize this in any way? It would have been a fairly important story and should have been pretty easy for somebody else to reproduce by generating the same completions and comparing them to your commit history. I don’t mean to be dismissive but unless you did something like that or documented it in some way it’s hard to give too much weight to a “source: trust me” comment from a brand-new HN account.
    - sharminglkw 3y ago
      I actually posted it on HN! If you look hard enough you might even find it. Just don't dox me, thanks.
  - IshKebab 3y ago
    Bullshit. That would be global news.
    - sharminglkw 3y ago
      [flagged]