3 ms·

In my experience it's more generative. It responds to variable names in my code, formatting, patterns in how I'm writing (am I using ES6 features? async/await o

by cdrini 4y ago

In my experience it's more generative. It responds to variable names in my code, formatting, patterns in how I'm writing (am I using ES6 features? async/await or raw promises? Etc). Granted it could still be copying, but when it's generating usually less than 5 lines at a time... I wonder if you could argue that all possible 5 line code snippets have already been written--wrt to like abstract logic/variable relationships/patterns. Also I think it's important to note that Copilot isn't an independent AI; it's a human in the loop system. In my experience I always make adjustments to the code it generates to better fit my needs. So on top of the transformation that copilot does, I'm also stacking transformation on top of that. So the final, potentially copyrightable product is very far removed from the training data. Copilot's training algorithm, OpenAI Codex, "is a descendant of GPT-3" under the hood. I think code is unique in that unlike text and images, there's much less variance in code. It's not as expressive as English, or visual art. So I think there might be a higher chance it'll generate something similar to its training data--but only because all code is inherently more similar.

toss1 4y ago
Thanks - very helpful!! Good points about you adding transformations on top of the Copilot output and less variation in code due to it's structured nature. Interesting also that it responds to variable names and patterns in your writing/code. Does it also respond to comments? Your answer points me more towards the not-infringing argument. Perhaps the best solution would be to have a companion plagiarism-checker tool that examined your code vs it's training set of GPL/MIT licensed code when you are nearly finished to flag significant copying. Shouldn't be too hard and would avoid the whole problem (and also maybe sometimes point you to a library you should be using instead of rolling your own).
- cdrini 4y ago
  Thanks for listening! The comment thread for anything copilot related is generally super polarised, and I really appreciated your calm comment and your calm response :) If I understand your question correctly, it does respond to comments. One of the best ways to interact with copilot is to write a comment (eg "Read input until y/n is entered") and have it generate the resulting code. If you're asking if it matches commenting style, I think it does, but I haven't pushed it in that direction too much. I think that's an interesting proposal, although it does place the onus on the developer. I would be curious to know how often developers would fail that check even without Copilot! Overall I think there are definitely open questions, but I'm personally really excited by systems like GPT, Copilot, or Dallee. The future is not clear, but I think these tools to some extent make the internet make sense. There's way too much data online to make sense of--be it code, text, or images. Unlike a search engine which just links to hopefully relevant material, these tools "learn" from all that data and respond with an answer of sorts--not a list of references. I think one huge improvement would be making these systems more explainable. So getting a certain response, you can also see the thousands of references that were used to generate that response. That would help a lot in providing transparency in whether there is plagiarism happening, and also just be an immensely useful tool for humanity and the internet. It feels like the next logical step for the internet. I would even say it feels like the internet was built for this!
  - toss1 4y ago
    Thx for the kind words and helping me get a better understanding! I think you are really onto something about GPT/Copilot/DALL-E this being almost what the Internet was built for, or at least one of the fundamental growth stages - first communicate, then accumulate data/info online, then make it searchable, then start automatically "understanding" synthesizing it... (with the quotes around "understanding" doing a lot of work). And a good point you make about not relying on the programmers to run the plagiarism checks - the tool should do it itself. It sounds like this is indeed closer to an copyright-OK generating of new code, rather than mere laundering, and if it isn't yet there, it seems like the copy/paste paradigm would be a juvenile phase, and it should improve and get more "creative" and less copy/paste-ish with further development.

by cdrini 4y ago

toss1 4y ago

Thanks - very helpful!! Good points about you adding transformations on top of the Copilot output and less variation in code due to it's structured nature. Interesting also that it responds to variable names and patterns in your writing/code. Does it also respond to comments? Your answer points me more towards the not-infringing argument. Perhaps the best solution would be to have a companion plagiarism-checker tool that examined your code vs it's training set of GPL/MIT licensed code when you are nearly finished to flag significant copying. Shouldn't be too hard and would avoid the whole problem (and also maybe sometimes point you to a library you should be using instead of rolling your own).

cdrini 4y ago
Thanks for listening! The comment thread for anything copilot related is generally super polarised, and I really appreciated your calm comment and your calm response :) If I understand your question correctly, it does respond to comments. One of the best ways to interact with copilot is to write a comment (eg "Read input until y/n is entered") and have it generate the resulting code. If you're asking if it matches commenting style, I think it does, but I haven't pushed it in that direction too much. I think that's an interesting proposal, although it does place the onus on the developer. I would be curious to know how often developers would fail that check even without Copilot! Overall I think there are definitely open questions, but I'm personally really excited by systems like GPT, Copilot, or Dallee. The future is not clear, but I think these tools to some extent make the internet make sense. There's way too much data online to make sense of--be it code, text, or images. Unlike a search engine which just links to hopefully relevant material, these tools "learn" from all that data and respond with an answer of sorts--not a list of references. I think one huge improvement would be making these systems more explainable. So getting a certain response, you can also see the thousands of references that were used to generate that response. That would help a lot in providing transparency in whether there is plagiarism happening, and also just be an immensely useful tool for humanity and the internet. It feels like the next logical step for the internet. I would even say it feels like the internet was built for this!
- toss1 4y ago
  Thx for the kind words and helping me get a better understanding! I think you are really onto something about GPT/Copilot/DALL-E this being almost what the Internet was built for, or at least one of the fundamental growth stages - first communicate, then accumulate data/info online, then make it searchable, then start automatically "understanding" synthesizing it... (with the quotes around "understanding" doing a lot of work). And a good point you make about not relying on the programmers to run the plagiarism checks - the tool should do it itself. It sounds like this is indeed closer to an copyright-OK generating of new code, rather than mere laundering, and if it isn't yet there, it seems like the copy/paste paradigm would be a juvenile phase, and it should improve and get more "creative" and less copy/paste-ish with further development.