4 ms·

This model doesn't learn and abstract: it just pattern matches and replicates; that's why it was shown exactly replicating regions of code--long enough to not b

by saurik 5y ago

This model doesn't learn and abstract: it just pattern matches and replicates; that's why it was shown exactly replicating regions of code--long enough to not be "de minimis" and recognizable enough to include the comments--that happen to be popular... which would be fine, as long as the license on said code were also being replicated. It just isn't reasonable to try to pretend Copilot--or GPT-3 in general--is some kind of general purpose AI worthy of being compared with the fair use rights of a human learning techniques: this is a machine learning model that likes to copy/paste not just tiny bits of code but entire functions out of other peoples' projects, and most of what makes it fancy is that it is good at adapting what it copies to the surrounding conditions.

the8472 5y ago
This is called prompt engineering. If you find a popular, frequently repeated code snippet and then fashion a prompt that is tailored to that snippet then yes the NN will recite it verbatim like a poem. But that doesn't mean it's the only thing it does or even that it does it frequently. It's like calling a human a parrot because he completed a line from a famous poem when the previous speaker left it unfinished. The same argument was brought up with GPT too and has been long debunked. The authors (and others) checked samples against the training corpus and it only rarely copies unless you prod it to.
- saurik 5y ago
  I don't know if I agree with your argument about GPT-3, but I think our disagreement seems to be besides the point: if your human parrot did that, they would--not just in theory but in actual fact! see all the cases of this in the music industry--get sued for it, even if they claim they didn't mean to and it was merely a really entrenched memory.
  - the8472 5y ago
    The point is that many of the examples you see are intentional, through prompt engineering. The pilot asked the copilot to violate copyright, the copilot complied. Don't blame the copilot. There also are cases where this happens unintentionally, but those are not the norm.
moyix 5y ago
Have you used Copilot? I have not, but I have trained a GPT2 model on open source projects (https://doesnotexist.codes/ https://doesnotexist.codes/). It does not just pattern match and replicate. It can be cajoled into reproducing some memorized snippets, but this is not the norm; in my experience the vast majority of what it generates is novel. The exceptions are extremely popular snippets that are repeated many many times in the training data, like license boilerplate. Perhaps Copilot behaves very differently from my own model, but I strongly suspect that the examples that have been going around twitter are outliers. Github's study agrees: https://docs.github.com/en/github/copilot/research-recitation https://docs.github.com/en/github/copilot/research-recitatio... (though of course this should be replicated independently).
- saurik 5y ago
  So, to verify, your claim is that GPT-3, when trained on a corpus of human text, isn't merely managing to string together a bunch of high-probability sequences of symbol constructs--which is how every article I have ever read on how it functions describes the technology--but is instead managing to build a model of the human world and the mechanism of narration required to describe it, with which it uses to write new prose... a claim you must make in order to then argue that GPT-3 works like a human engineer learning a model of computers, libraries, and engineering principals from which it can then write code, instead of merely using pattern recognition as I stated? As someone who spent years studying graduate linguistics and cognitive science (though admittedly 15-20 years ago, so I certainly haven't studied this model: I have only read about it occasionally in passing) I frankly think you are just trying to conflate levels of understanding, in order to make GPT-3 sound more magical than it is :/.
  - moyix 5y ago
    What? I don't think I made any claim of the sort. I'm claiming that it does more than mere regurgitation and has done some amount of abstraction, not that it has human-level understanding. As an example, GPT-3 learned some arithmetic and can solve basic math problems not in its training set. This is beyond pattern matching and replication, IMO. I'm not really sure why we should consider Copilot legally different from a fancy pen – if you use it to write infringing code then that's infringement by the user, not the pen. This leaves the practical question of how often it will do so, and my impression is that it's not often.
visarga 5y ago
> this is a machine learning model that likes to copy/paste not just tiny bits of code but entire functions out of other peoples' projects Github could make a blacklist and tell Copilot never to suggest that code. Problem solved. You use one of the other 9 suggestions.
robbedpeter 5y ago
Transformers do learn and abstract. Not as well as humans, but for whatever definitive of innovation or creativity you wanna run with, these gpt models have it. It's not magic, it's math, but these programs are approximating the human function of media synthesis across narrowly limited domains. These aren't your crazy uncle's Markov chain chatbots. They're sophisticated bayesian models trained to approximate the functions that produced the content used in training.
- astrange 5y ago
  Are they Bayesian? It would be good if ML models were Bayesian (they'd be able to show uncertainty better) but they usually aren't.
  - robbedpeter 5y ago
    The model and attention mechanism produces Bayesian properties, but transformers as a whole contain non-Bayesian aspects, depending on how rigorous you want to be in defining Bayesian.

by saurik 5y ago

the8472 5y ago

This is called prompt engineering. If you find a popular, frequently repeated code snippet and then fashion a prompt that is tailored to that snippet then yes the NN will recite it verbatim like a poem. But that doesn't mean it's the only thing it does or even that it does it frequently. It's like calling a human a parrot because he completed a line from a famous poem when the previous speaker left it unfinished. The same argument was brought up with GPT too and has been long debunked. The authors (and others) checked samples against the training corpus and it only rarely copies unless you prod it to.

saurik 5y ago
I don't know if I agree with your argument about GPT-3, but I think our disagreement seems to be besides the point: if your human parrot did that, they would--not just in theory but in actual fact! see all the cases of this in the music industry--get sued for it, even if they claim they didn't mean to and it was merely a really entrenched memory.
- the8472 5y ago
  The point is that many of the examples you see are intentional, through prompt engineering. The pilot asked the copilot to violate copyright, the copilot complied. Don't blame the copilot. There also are cases where this happens unintentionally, but those are not the norm.

moyix 5y ago

Have you used Copilot? I have not, but I have trained a GPT2 model on open source projects (https://doesnotexist.codes/ https://doesnotexist.codes/). It does not just pattern match and replicate. It can be cajoled into reproducing some memorized snippets, but this is not the norm; in my experience the vast majority of what it generates is novel. The exceptions are extremely popular snippets that are repeated many many times in the training data, like license boilerplate. Perhaps Copilot behaves very differently from my own model, but I strongly suspect that the examples that have been going around twitter are outliers. Github's study agrees: https://docs.github.com/en/github/copilot/research-recitation https://docs.github.com/en/github/copilot/research-recitatio... (though of course this should be replicated independently).

saurik 5y ago
So, to verify, your claim is that GPT-3, when trained on a corpus of human text, isn't merely managing to string together a bunch of high-probability sequences of symbol constructs--which is how every article I have ever read on how it functions describes the technology--but is instead managing to build a model of the human world and the mechanism of narration required to describe it, with which it uses to write new prose... a claim you must make in order to then argue that GPT-3 works like a human engineer learning a model of computers, libraries, and engineering principals from which it can then write code, instead of merely using pattern recognition as I stated? As someone who spent years studying graduate linguistics and cognitive science (though admittedly 15-20 years ago, so I certainly haven't studied this model: I have only read about it occasionally in passing) I frankly think you are just trying to conflate levels of understanding, in order to make GPT-3 sound more magical than it is :/.
- moyix 5y ago
  What? I don't think I made any claim of the sort. I'm claiming that it does more than mere regurgitation and has done some amount of abstraction, not that it has human-level understanding. As an example, GPT-3 learned some arithmetic and can solve basic math problems not in its training set. This is beyond pattern matching and replication, IMO. I'm not really sure why we should consider Copilot legally different from a fancy pen – if you use it to write infringing code then that's infringement by the user, not the pen. This leaves the practical question of how often it will do so, and my impression is that it's not often.

visarga 5y ago

> this is a machine learning model that likes to copy/paste not just tiny bits of code but entire functions out of other peoples' projects Github could make a blacklist and tell Copilot never to suggest that code. Problem solved. You use one of the other 9 suggestions.

robbedpeter 5y ago

Transformers do learn and abstract. Not as well as humans, but for whatever definitive of innovation or creativity you wanna run with, these gpt models have it. It's not magic, it's math, but these programs are approximating the human function of media synthesis across narrowly limited domains. These aren't your crazy uncle's Markov chain chatbots. They're sophisticated bayesian models trained to approximate the functions that produced the content used in training.

astrange 5y ago
Are they Bayesian? It would be good if ML models were Bayesian (they'd be able to show uncertainty better) but they usually aren't.
- robbedpeter 5y ago
  The model and attention mechanism produces Bayesian properties, but transformers as a whole contain non-Bayesian aspects, depending on how rigorous you want to be in defining Bayesian.