5 ms·
This seems to be what people imagine about it, not what it actually does, although I don’t doubt you could cherry-pick some snippet after a lot of trial and err
by natch 4y ago
This seems to be what people imagine about it, not what it actually does, although I don’t doubt you could cherry-pick some snippet after a lot of trial and error to try to claim that it had regurgitated something verbatim. But certainly let’s see the examples.
- bombolo 4y agoYou never know if a snippet it created came from another project verbatim or not… unless you claim you know all of the code that exists?
- bqmjjx0kac 4y agoThat's a bit extreme. In theory, an LLM's proclivity for plagiarism could be studied by testing it with various prompts and searching its training data for its responses (maybe with some edit distance tolerance).
- bombolo 4y agoExcept the training data is secret…
- natch 4y agoYou can search github and other open sources to find at least a likely subset of the training data though.
- bombolo 4y agoYou suggest doing this, by hand, for every suggestion?
- natch 4y agoJust try it out for some code you have on github where you know yours is the only solution out there. You'll be pleasantly surprised to see that it does not suggest a verbatim copy/paste of your code or anything close to it, unless you try this with a one liner like how to do an fopen(), which would not be a good test, and would not be the only solution out there. And then seeing the result, you can adjust your theory. So, in short, I suggest simply testing your theory, not anything absurd like what you are coming up with.
- bombolo 4y agoWhat would that prove? I still have no access to all the proprietary code generated from copilot, and no idea if it did copy paste or not in all those cases. You suggest I try it twice and since it will probably not copy paste in those 2 tries, assume it never copy pastes (despite existing evidence that it does copy paste in some other cases). What problem would this exercise solve? I can't see it.
- natch 4y agoSearch tools are a thing. Grep, Google, Github, etc. But yes, exactly, you never know if a snippet came from another project or not, so let's not assume it did without some convincing evidence.
- bombolo 4y agoWhy assume it didn't?
- natch 4y agoI’ve done tests and it passed with flying colors so it’s not an assumption. So the premise of your question is flawed.
- bombolo 4y agoIt has been shown that github copilot does copy paste. The fact that you tried it a couple of times (or 10 or 20) means absolutely nothing. 1 copyright infringement is enough for a lawsuit.