4 ms·

Yes, people can read my open-sourced code and learn from it, like they can do with paints, movies, sculpts, and books. No, I don't allow to copy my code freely

by drran 5y ago

Yes, people can read my open-sourced code and learn from it, like they can do with paints, movies, sculpts, and books. No, I don't allow to copy my code freely. Can you explain, what point you are trying to defend here?

FeepingCreature 5y ago
There are cases where the network can reproduce code verbatim, just like there are some functions that a good programmer may know by heart. However, this is not how Copilot normally works; it acquires understanding of code, not a copy-paste library. Well, that's overstating things: it's kinda an open question under what circumstances GPT will learn text by heart. However, it seems to be extremely rare. (Human creators also sometimes reproduce things they've read verbatim, without noticing.)
- swiftcoder 5y ago
  > it acquires understanding of code This is somewhat debatable. Would you consider a Markov chain to have an understanding of the input data? It too sometimes reproduces training data verbatim, but mostly produce remixes of many short snippets instead. On the spectrum from Markov chain to Human comprehension, I'd submit that GPT is much closer to the former than the latter.
  - FeepingCreature 5y ago
    I would consider a Markov chain to have "the sort of thing that understanding is a more complex version of" of the input data. Markov chains compress inputs. Understanding is compression. So I'm willing to bite that bullet. I submit that GPT is much closer to the latter than the former, and may even overtake it in its domain. It lacks hardware that the human brain uses to reflect on its knowledge and understand what it is thinking - but just as an "overtuned intuition", just like how AlphaZero's network forms a quite capable chess AI without even using tree search, just by "taking the move that feels right to it", I think it's on par with human intuition, or even stronger, because it cannot rely on reflection like humans can. (In other words, I think there isn't a spectrum of Markov chains to human comprehension. Human comprehension is a different kind of system. However, there is a spectrum of Markov chains to human intuition.)
IshKebab 5y ago
The point is that if you are happy allowing people to learn from your code, you should be happy allowing an AI to learn from it. After learning from your code people may be able to write some of down verbatim (if they have a good memory). The same is true for AI. Just because they both can do that, does not mean that they will do it - Github is already working on informing users when suggested code is a verbatim copy. So in conclusion, it's unreasonable to object to your code being used for training an AI when you are ok with using it to train a person. It isn't a valid objection that the AI has learnt some snippets by heart, because people can also do that when reading your code.

by drran 5y ago

FeepingCreature 5y ago

There are cases where the network can reproduce code verbatim, just like there are some functions that a good programmer may know by heart. However, this is not how Copilot normally works; it acquires understanding of code, not a copy-paste library. Well, that's overstating things: it's kinda an open question under what circumstances GPT will learn text by heart. However, it seems to be extremely rare. (Human creators also sometimes reproduce things they've read verbatim, without noticing.)

swiftcoder 5y ago
> it acquires understanding of code This is somewhat debatable. Would you consider a Markov chain to have an understanding of the input data? It too sometimes reproduces training data verbatim, but mostly produce remixes of many short snippets instead. On the spectrum from Markov chain to Human comprehension, I'd submit that GPT is much closer to the former than the latter.
- FeepingCreature 5y ago
  I would consider a Markov chain to have "the sort of thing that understanding is a more complex version of" of the input data. Markov chains compress inputs. Understanding is compression. So I'm willing to bite that bullet. I submit that GPT is much closer to the latter than the former, and may even overtake it in its domain. It lacks hardware that the human brain uses to reflect on its knowledge and understand what it is thinking - but just as an "overtuned intuition", just like how AlphaZero's network forms a quite capable chess AI without even using tree search, just by "taking the move that feels right to it", I think it's on par with human intuition, or even stronger, because it cannot rely on reflection like humans can. (In other words, I think there isn't a spectrum of Markov chains to human comprehension. Human comprehension is a different kind of system. However, there is a spectrum of Markov chains to human intuition.)

IshKebab 5y ago

The point is that if you are happy allowing people to learn from your code, you should be happy allowing an AI to learn from it. After learning from your code people may be able to write some of down verbatim (if they have a good memory). The same is true for AI. Just because they both can do that, does not mean that they will do it - Github is already working on informing users when suggested code is a verbatim copy. So in conclusion, it's unreasonable to object to your code being used for training an AI when you are ok with using it to train a person. It isn't a valid objection that the AI has learnt some snippets by heart, because people can also do that when reading your code.