3 ms·

Nice theory. Won’t work out in practice, because this produces code that will run, and it’s AI, so it must be good, right? When you found code on the internet,

Nice theory. Won’t work out in practice, because this produces code that will run, and it’s AI, so it must be good, right? When you found code on the internet, it was presented in a context that let you make better judgement (e.g. on Stack Overflow this regular expression would have had a score of roughly −∞ and multiple highly-voted comments saying “do not use this, it’s catastrophically bad”), and where you have to put in more effort to plug it in and shuffle things around a bit as well. With Copilot, you get given ready-to-go code without any sanity checking at all. See even how, a few seconds later in the video, the author does test it out—but not thoroughly enough.

visarga 5y ago
I think Copilot should report the matching source URL to allow the user to visit the page and see the context and license. This move would also placate some copyright questions because it would be like searching StackOverflow or Github for inspiration. The problem of content attribution (exact and fuzzy match) has been studied before under the task of plagiarism detection for student essays. Funny thing is that a plagiarism detection Copilot would also disclose past cases of copyright violation and cause attribution disputes because code sitting unchecked in various repos would suddenly become visible.
- duskwuff 5y ago
  > I think Copilot should report the matching source URL That's the problem. The output of a GAN like Copilot usually can't be traced directly back to a single input.
  - FractalHQ 5y ago
    A list sorted by probability would be better than nothing. Would a GAN like Copilot be able to provide that?
    - duskwuff 5y ago
      Not easily. It's a rather opaque process.
      - virgilp 5y ago
        And besides, let's get real, nobody would look at the links - it defeats the purpose of copilot.
  - visarga 5y ago
    If you can't trace the source then it's transformative use. If it matches training data then it needs to report the source like a search engine and place all responsibility on the user. And fuzzy code matching could be easily implemented by using the a model similar to CLIP (contrastive) to embed code snippets.
    - krainboltgreene 5y ago
      > If you can't trace the source then it's transformative use. That's not how "transformative use" works.
bravetraveler 5y ago
Decently puts my feelings towards this whole thing Aside from the beaten horse concerns like licensing... I worry about the training we're giving ourselves and future generations The upfront presentation of 'suggestions' skews the perception, a fair bit of 'no warranty guaranteed' comes from having to go dig it up

visarga 5y ago

I think Copilot should report the matching source URL to allow the user to visit the page and see the context and license. This move would also placate some copyright questions because it would be like searching StackOverflow or Github for inspiration. The problem of content attribution (exact and fuzzy match) has been studied before under the task of plagiarism detection for student essays. Funny thing is that a plagiarism detection Copilot would also disclose past cases of copyright violation and cause attribution disputes because code sitting unchecked in various repos would suddenly become visible.

duskwuff 5y ago
> I think Copilot should report the matching source URL That's the problem. The output of a GAN like Copilot usually can't be traced directly back to a single input.
- FractalHQ 5y ago
  A list sorted by probability would be better than nothing. Would a GAN like Copilot be able to provide that?
  - duskwuff 5y ago
    Not easily. It's a rather opaque process.
    - virgilp 5y ago
      And besides, let's get real, nobody would look at the links - it defeats the purpose of copilot.
- visarga 5y ago
  If you can't trace the source then it's transformative use. If it matches training data then it needs to report the source like a search engine and place all responsibility on the user. And fuzzy code matching could be easily implemented by using the a model similar to CLIP (contrastive) to embed code snippets.
  - krainboltgreene 5y ago
    > If you can't trace the source then it's transformative use. That's not how "transformative use" works.

bravetraveler 5y ago

Decently puts my feelings towards this whole thing Aside from the beaten horse concerns like licensing... I worry about the training we're giving ourselves and future generations The upfront presentation of 'suggestions' skews the perception, a fair bit of 'no warranty guaranteed' comes from having to go dig it up