4 ms·

It can produce code snippets that were never seen by generating fragments from various sources and combining them in a new way. This makes it different from a s

It can produce code snippets that were never seen by generating fragments from various sources and combining them in a new way. This makes it different from a search engine, which only returns existing items.

bogwog 5y ago
Is it producing code (by which I mean creating/inventing new code by itself), or is it just combining existing code? Because to me it seems like the latter is a more appropriate description. * AI searches for code in its neural-net-encoded database using your search terms (ex: "fast inverse square root") * AI parses and generates AST from the snippet it found * AI parses and generates AST from your existing codebase * AI merges the ASTs in a way that compiles (it inserts snippet at your cursor, renames variables/function/class names to match existing ones in your program, etc) * AI converts AST back into source code Is AI intelligently producing new code in that example? Because I don't think it is. What would be an interesting test of whether it can actually generate code is if it were tasked with implementing a new algorithm that isn't in the training set at all, and could not possibly be implemented by simply merging existing code snippets together. Maybe by describing a detailed imaginary protocol that does nothing useful, but requires some complicated logic, abstract concepts, and math. A person can implement an algorithm they've never seen before by applying critical thinking and creativity (and maybe domain knowledge). If an AI can't do that, then you cannot credibly say that it's writing original code, because the only thing it has ever read, and the only thing it will ever write, is other people's code.
- LightMachine 5y ago
  That isn't even necessary. I've been exploring GPT-3 for a while and it is completely incapable of any reasoning. If you enter short unique logical sentences like "Bob had 5 apples, gave 2 to Mary, then ate the same amount. How many apples Bob has left?" No matter how many previous examples you give it (to be sure it gets the question), it gets it wrong. It is simply incapable of reasoning about what is going on.
- joshuamorton 5y ago
  > Is it producing code (by which I mean creating/inventing new code by itself), or is it just combining existing code? How do you differentiate between these two things? If I take `for i in range(10):` from one place and `print(x * 2) from another place, and combine them to get for i in range(10): print(i ** 2) have I "produced" something or "combined" existing things? As an aside, your understanding of how the model works here is completely wrong. Like, just absolutely fundamentally completely wrong.
  - bogwog 5y ago
    > How do you differentiate between these two things? That's a contrived example because none of those lines could be protected by copyright, patents, etc. A better example might be if you started selling a 30 minute movie that was just the first 15 minutes of Toy Story spliced together with the last 15 minutes of Shrek. I'm not a lawyer, but I'm pretty sure that would qualify as a derivative work, meaning you're potentially infringing on someone's rights (unless they've given you permission/a license). And to be clear, none of these problems are new. People have been fighting over copyright and it's philosophy in court for a very long time. The only thing that's different here is that it seems some people think it's ok to ignore copyright if you use Copilot as a proxy for the infringement. > As an aside, your understanding of how the model works here is completely wrong. Like, just absolutely fundamentally completely wrong. Of course I don't, it's a neural network. You don't know either. That example I posted could be exactly what it's doing, or not even close. (although for the record, I wasn't trying to explain how copilot works in that comment. It was a hypothetical "AI" for the sake of discussion, not that it matters. My point about it being copyright infringement is the same even if that hypothetical implementation is wrong)
    - joshuamorton 5y ago
      > That's a contrived example because none of those lines could be protected by copyright, patents, etc. Well no. The question I'm asking about, the philosophical distinction between "producing" or "combining" is a valid question no matter the copyrightability of anything. It's an interesting philosophical question even if we presume that copyright is bubkis. > It was a hypothetical "AI" for the sake of discussion, not that it matters. Ah, my mistake. I see that now. > Of course I don't, it's a neural network. You don't know either. I may not know how to make a lightbulb, but I do know hundreds of ways not to make one ;)
- nl 5y ago
  > A person can implement an algorithm they've never seen before by applying critical thinking and creativity (and maybe domain knowledge). If an AI can't do that, then you cannot credibly say that it's writing original code, because the only thing it has ever read, and the only thing it will ever write, is other people's code. This doesn't hold at all. Not many people can come up with an original sorting algorithm for example, but people write code all the time.
- nl 5y ago
  This understanding of how a generative process from a NN works is completely wrong. There is no database lookup. I've attempted to break that part down here: https://news.ycombinator.com/item?id=27744156 https://news.ycombinator.com/item?id=27744156 But you seem to have a basic fundamental misunderstanding of what is going on inside the NN. There is no "search for code" - it is generating new code each time, but sometimes that code will be the same as something it has seen because there is little or no variation in the training data for that snippet. The NN generates code token by token, conditioned on the code leading up to it (and perhaps the code ahead, similar to BERT). If you see tokens like this you probably generate the same next token too: for i in range(1,10) You have conditioned your input on the code you have seen and the most likely token you produce is ":". That's what the NN does, but for much longer range conditioning.
  - throwaway413 5y ago
    I am not educated on this matter, but I have to ask for your clarification. Would that not be just a pre-emptive lookup? Akin to keeping a cache of "results" per input token that are essentially memorized and regurgitated? Sounds like there is still a db lookup, just not at runtime and instead at build time of the NN. Can you clarify this please?
    - freeone3000 5y ago
      GPT-3's raw output is "logits", or indexes into an encoding space. The encoding space contains individual tokens; for generation, it would be words, or even word pieces. The pieces are as small as "for", or "if". Constructing code from an embedding space, even if it is more specialized, is like constructing sentences by using a dictionary -- it is a lookup table, but it's not a database. Generation works by looking at the existing document (or portion), and based on what is already present, generating a token. Then repeating until some condition is met (such as length, end of sentence, something else). The issue here is that certain sentences (code segments) are memorized, and reproduced -- much like a language learner who completes every sentence which begins with "Mi nombre" with the phrase "Mi nombre es Mark". The regurgitation is based on high probability built into the priors, not an explicit lookup. A different logit sampling method (instead of taking the likeliest) reduces regurgitation, without changing anything else about the network. (It also makes nonsense happen more often, since nonsense items are inherently less likely!)
eitland 5y ago
The fact it reproduce code verbatim, including comments and even swear words means it is definitely copying some of the time. Does it copy all the time? Doesn't matter. Plagiarism is plagiarism even regardless of it is done by a student in school, an author, a monkey or an "AI". You wouldn't accept this from a student, you shouldn't accept it from a coworker (unless you are releasing under a compatible license) and of course you should accept it from Microsoft.

bogwog 5y ago

Is it producing code (by which I mean creating/inventing new code by itself), or is it just combining existing code? Because to me it seems like the latter is a more appropriate description. * AI searches for code in its neural-net-encoded database using your search terms (ex: "fast inverse square root") * AI parses and generates AST from the snippet it found * AI parses and generates AST from your existing codebase * AI merges the ASTs in a way that compiles (it inserts snippet at your cursor, renames variables/function/class names to match existing ones in your program, etc) * AI converts AST back into source code Is AI intelligently producing new code in that example? Because I don't think it is. What would be an interesting test of whether it can actually generate code is if it were tasked with implementing a new algorithm that isn't in the training set at all, and could not possibly be implemented by simply merging existing code snippets together. Maybe by describing a detailed imaginary protocol that does nothing useful, but requires some complicated logic, abstract concepts, and math. A person can implement an algorithm they've never seen before by applying critical thinking and creativity (and maybe domain knowledge). If an AI can't do that, then you cannot credibly say that it's writing original code, because the only thing it has ever read, and the only thing it will ever write, is other people's code.

LightMachine 5y ago
That isn't even necessary. I've been exploring GPT-3 for a while and it is completely incapable of any reasoning. If you enter short unique logical sentences like "Bob had 5 apples, gave 2 to Mary, then ate the same amount. How many apples Bob has left?" No matter how many previous examples you give it (to be sure it gets the question), it gets it wrong. It is simply incapable of reasoning about what is going on.
joshuamorton 5y ago
> Is it producing code (by which I mean creating/inventing new code by itself), or is it just combining existing code? How do you differentiate between these two things? If I take `for i in range(10):` from one place and `print(x * 2) from another place, and combine them to get for i in range(10): print(i ** 2) have I "produced" something or "combined" existing things? As an aside, your understanding of how the model works here is completely wrong. Like, just absolutely fundamentally completely wrong.
- bogwog 5y ago
  > How do you differentiate between these two things? That's a contrived example because none of those lines could be protected by copyright, patents, etc. A better example might be if you started selling a 30 minute movie that was just the first 15 minutes of Toy Story spliced together with the last 15 minutes of Shrek. I'm not a lawyer, but I'm pretty sure that would qualify as a derivative work, meaning you're potentially infringing on someone's rights (unless they've given you permission/a license). And to be clear, none of these problems are new. People have been fighting over copyright and it's philosophy in court for a very long time. The only thing that's different here is that it seems some people think it's ok to ignore copyright if you use Copilot as a proxy for the infringement. > As an aside, your understanding of how the model works here is completely wrong. Like, just absolutely fundamentally completely wrong. Of course I don't, it's a neural network. You don't know either. That example I posted could be exactly what it's doing, or not even close. (although for the record, I wasn't trying to explain how copilot works in that comment. It was a hypothetical "AI" for the sake of discussion, not that it matters. My point about it being copyright infringement is the same even if that hypothetical implementation is wrong)
  - joshuamorton 5y ago
    > That's a contrived example because none of those lines could be protected by copyright, patents, etc. Well no. The question I'm asking about, the philosophical distinction between "producing" or "combining" is a valid question no matter the copyrightability of anything. It's an interesting philosophical question even if we presume that copyright is bubkis. > It was a hypothetical "AI" for the sake of discussion, not that it matters. Ah, my mistake. I see that now. > Of course I don't, it's a neural network. You don't know either. I may not know how to make a lightbulb, but I do know hundreds of ways not to make one ;)
nl 5y ago
> A person can implement an algorithm they've never seen before by applying critical thinking and creativity (and maybe domain knowledge). If an AI can't do that, then you cannot credibly say that it's writing original code, because the only thing it has ever read, and the only thing it will ever write, is other people's code. This doesn't hold at all. Not many people can come up with an original sorting algorithm for example, but people write code all the time.
nl 5y ago
This understanding of how a generative process from a NN works is completely wrong. There is no database lookup. I've attempted to break that part down here: https://news.ycombinator.com/item?id=27744156 https://news.ycombinator.com/item?id=27744156 But you seem to have a basic fundamental misunderstanding of what is going on inside the NN. There is no "search for code" - it is generating new code each time, but sometimes that code will be the same as something it has seen because there is little or no variation in the training data for that snippet. The NN generates code token by token, conditioned on the code leading up to it (and perhaps the code ahead, similar to BERT). If you see tokens like this you probably generate the same next token too: for i in range(1,10) You have conditioned your input on the code you have seen and the most likely token you produce is ":". That's what the NN does, but for much longer range conditioning.
- throwaway413 5y ago
  I am not educated on this matter, but I have to ask for your clarification. Would that not be just a pre-emptive lookup? Akin to keeping a cache of "results" per input token that are essentially memorized and regurgitated? Sounds like there is still a db lookup, just not at runtime and instead at build time of the NN. Can you clarify this please?
  - freeone3000 5y ago
    GPT-3's raw output is "logits", or indexes into an encoding space. The encoding space contains individual tokens; for generation, it would be words, or even word pieces. The pieces are as small as "for", or "if". Constructing code from an embedding space, even if it is more specialized, is like constructing sentences by using a dictionary -- it is a lookup table, but it's not a database. Generation works by looking at the existing document (or portion), and based on what is already present, generating a token. Then repeating until some condition is met (such as length, end of sentence, something else). The issue here is that certain sentences (code segments) are memorized, and reproduced -- much like a language learner who completes every sentence which begins with "Mi nombre" with the phrase "Mi nombre es Mark". The regurgitation is based on high probability built into the priors, not an explicit lookup. A different logit sampling method (instead of taking the likeliest) reduces regurgitation, without changing anything else about the network. (It also makes nonsense happen more often, since nonsense items are inherently less likely!)

eitland 5y ago

The fact it reproduce code verbatim, including comments and even swear words means it is definitely copying some of the time. Does it copy all the time? Doesn't matter. Plagiarism is plagiarism even regardless of it is done by a student in school, an author, a monkey or an "AI". You wouldn't accept this from a student, you shouldn't accept it from a coworker (unless you are releasing under a compatible license) and of course you should accept it from Microsoft.