4 ms·

> But it's also not exactly just a database. It contains contextual relationships as seen with things like GPT that are beyond what a typical database implement

by bogwog 5y ago

> But it's also not exactly just a database. It contains contextual relationships as seen with things like GPT that are beyond what a typical database implementation would be capable of. You mean in the same way that google.com isn't "just a database"? If Copilot isn't intelligent, then what makes it more special than a search engine? How is Copilot not just Limewire but for code? I could understand the argument that, if Copilot really is intelligent or sentient or something like that, then what it is producing is as original as what a human can produce (although, humans still have to respect copyright laws). However, I haven't seen anyone even attempt to make a serious argument like that.

freeone3000 5y ago
It can produce code snippets that were never seen by generating fragments from various sources and combining them in a new way. This makes it different from a search engine, which only returns existing items.
- bogwog 5y ago
  Is it producing code (by which I mean creating/inventing new code by itself), or is it just combining existing code? Because to me it seems like the latter is a more appropriate description. * AI searches for code in its neural-net-encoded database using your search terms (ex: "fast inverse square root") * AI parses and generates AST from the snippet it found * AI parses and generates AST from your existing codebase * AI merges the ASTs in a way that compiles (it inserts snippet at your cursor, renames variables/function/class names to match existing ones in your program, etc) * AI converts AST back into source code Is AI intelligently producing new code in that example? Because I don't think it is. What would be an interesting test of whether it can actually generate code is if it were tasked with implementing a new algorithm that isn't in the training set at all, and could not possibly be implemented by simply merging existing code snippets together. Maybe by describing a detailed imaginary protocol that does nothing useful, but requires some complicated logic, abstract concepts, and math. A person can implement an algorithm they've never seen before by applying critical thinking and creativity (and maybe domain knowledge). If an AI can't do that, then you cannot credibly say that it's writing original code, because the only thing it has ever read, and the only thing it will ever write, is other people's code.
  - LightMachine 5y ago
    That isn't even necessary. I've been exploring GPT-3 for a while and it is completely incapable of any reasoning. If you enter short unique logical sentences like "Bob had 5 apples, gave 2 to Mary, then ate the same amount. How many apples Bob has left?" No matter how many previous examples you give it (to be sure it gets the question), it gets it wrong. It is simply incapable of reasoning about what is going on.
  - joshuamorton 5y ago
    > Is it producing code (by which I mean creating/inventing new code by itself), or is it just combining existing code? How do you differentiate between these two things? If I take `for i in range(10):` from one place and `print(x * 2) from another place, and combine them to get for i in range(10): print(i ** 2) have I "produced" something or "combined" existing things? As an aside, your understanding of how the model works here is completely wrong. Like, just absolutely fundamentally completely wrong.
    - bogwog 5y ago
      > How do you differentiate between these two things? That's a contrived example because none of those lines could be protected by copyright, patents, etc. A better example might be if you started selling a 30 minute movie that was just the first 15 minutes of Toy Story spliced together with the last 15 minutes of Shrek. I'm not a lawyer, but I'm pretty sure that would qualify as a derivative work, meaning you're potentially infringing on someone's rights (unless they've given you permission/a license). And to be clear, none of these problems are new. People have been fighting over copyright and it's philosophy in court for a very long time. The only thing that's different here is that it seems some people think it's ok to ignore copyright if you use Copilot as a proxy for the infringement. > As an aside, your understanding of how the model works here is completely wrong. Like, just absolutely fundamentally completely wrong. Of course I don't, it's a neural network. You don't know either. That example I posted could be exactly what it's doing, or not even close. (although for the record, I wasn't trying to explain how copilot works in that comment. It was a hypothetical "AI" for the sake of discussion, not that it matters. My point about it being copyright infringement is the same even if that hypothetical implementation is wrong)
- eitland 5y ago
  The fact it reproduce code verbatim, including comments and even swear words means it is definitely copying some of the time. Does it copy all the time? Doesn't matter. Plagiarism is plagiarism even regardless of it is done by a student in school, an author, a monkey or an "AI". You wouldn't accept this from a student, you shouldn't accept it from a coworker (unless you are releasing under a compatible license) and of course you should accept it from Microsoft.
sjy 5y ago
Perhaps it’s not so different from a search engine like Google. The article cites Google’s successful defence, under US copyright law, of its practice of displaying ‘snippets’ from copyrighted books in search results. There is a clear difference between this and the distribution of complete copies on LimeWire.
nl 5y ago
Co-pilot produces original code (as in code that has never been written before). It's not just combining snippets. This should surprise no one who has seen the evolution of language models. Take a look at Kapathy's great write up from way back in 2015[1]. This generates Wikipedia syntax from a character-based RNN. It's operating on a per-character basis, and it doesn't have sufficient capacity to memorise large snippets. (The Paul Graham example spells this out: 1M characters in the dataset = 8M bits, and the network has 3.5M parameters). Semantic arguments about "is this intelligence?" I'll let others fight. [1] http://karpathy.github.io/2015/05/21/rnn-effectiveness/ http://karpathy.github.io/2015/05/21/rnn-effectiveness/
- refulgentis 5y ago
  You're swapping "It does, at least occasionally, combine snippets" for "it's just combining snippets"
  - nl 5y ago
    That is can combine snippets doesn't mean the OP's understanding of how the system works is correct. They seem to believe it is a database system. That's really not how this works, and the fact it behaves like one sometimes is disguising what it is doing. If I say "write a "for" loop from 0 to 10 in Python" probably 50% of implementations by Python programmers will look exactly the same. Some will be retrieving that from memory, but many will be using a generative process that generates the same code, because they've seen and done similar things thousands of times before. A neural network is doing a similar thing. "Write quicksort" makes it start generating tokens, and the loss function has optimised it to generate them in an order it has seen before. It's probably seen a decent number of variations of quicksort, so you might get a mix of what it has seen before. For other pieces of code it has only seen one implementation, so it will generate something very similar. There could be local variations (eg, it sees lots of loops, so it might use a different variation) but in general it will be very similar. But this isn't a database lookup function - it's generative against a loss function. This is subtle distinction, but it is reasonable that people on HN understand this.
    - 5y ago

by bogwog 5y ago

freeone3000 5y ago

It can produce code snippets that were never seen by generating fragments from various sources and combining them in a new way. This makes it different from a search engine, which only returns existing items.

bogwog 5y ago
Is it producing code (by which I mean creating/inventing new code by itself), or is it just combining existing code? Because to me it seems like the latter is a more appropriate description. * AI searches for code in its neural-net-encoded database using your search terms (ex: "fast inverse square root") * AI parses and generates AST from the snippet it found * AI parses and generates AST from your existing codebase * AI merges the ASTs in a way that compiles (it inserts snippet at your cursor, renames variables/function/class names to match existing ones in your program, etc) * AI converts AST back into source code Is AI intelligently producing new code in that example? Because I don't think it is. What would be an interesting test of whether it can actually generate code is if it were tasked with implementing a new algorithm that isn't in the training set at all, and could not possibly be implemented by simply merging existing code snippets together. Maybe by describing a detailed imaginary protocol that does nothing useful, but requires some complicated logic, abstract concepts, and math. A person can implement an algorithm they've never seen before by applying critical thinking and creativity (and maybe domain knowledge). If an AI can't do that, then you cannot credibly say that it's writing original code, because the only thing it has ever read, and the only thing it will ever write, is other people's code.
- LightMachine 5y ago
  That isn't even necessary. I've been exploring GPT-3 for a while and it is completely incapable of any reasoning. If you enter short unique logical sentences like "Bob had 5 apples, gave 2 to Mary, then ate the same amount. How many apples Bob has left?" No matter how many previous examples you give it (to be sure it gets the question), it gets it wrong. It is simply incapable of reasoning about what is going on.
- joshuamorton 5y ago
  > Is it producing code (by which I mean creating/inventing new code by itself), or is it just combining existing code? How do you differentiate between these two things? If I take `for i in range(10):` from one place and `print(x * 2) from another place, and combine them to get for i in range(10): print(i ** 2) have I "produced" something or "combined" existing things? As an aside, your understanding of how the model works here is completely wrong. Like, just absolutely fundamentally completely wrong.
  - bogwog 5y ago
    > How do you differentiate between these two things? That's a contrived example because none of those lines could be protected by copyright, patents, etc. A better example might be if you started selling a 30 minute movie that was just the first 15 minutes of Toy Story spliced together with the last 15 minutes of Shrek. I'm not a lawyer, but I'm pretty sure that would qualify as a derivative work, meaning you're potentially infringing on someone's rights (unless they've given you permission/a license). And to be clear, none of these problems are new. People have been fighting over copyright and it's philosophy in court for a very long time. The only thing that's different here is that it seems some people think it's ok to ignore copyright if you use Copilot as a proxy for the infringement. > As an aside, your understanding of how the model works here is completely wrong. Like, just absolutely fundamentally completely wrong. Of course I don't, it's a neural network. You don't know either. That example I posted could be exactly what it's doing, or not even close. (although for the record, I wasn't trying to explain how copilot works in that comment. It was a hypothetical "AI" for the sake of discussion, not that it matters. My point about it being copyright infringement is the same even if that hypothetical implementation is wrong)
eitland 5y ago
The fact it reproduce code verbatim, including comments and even swear words means it is definitely copying some of the time. Does it copy all the time? Doesn't matter. Plagiarism is plagiarism even regardless of it is done by a student in school, an author, a monkey or an "AI". You wouldn't accept this from a student, you shouldn't accept it from a coworker (unless you are releasing under a compatible license) and of course you should accept it from Microsoft.

sjy 5y ago

Perhaps it’s not so different from a search engine like Google. The article cites Google’s successful defence, under US copyright law, of its practice of displaying ‘snippets’ from copyrighted books in search results. There is a clear difference between this and the distribution of complete copies on LimeWire.

nl 5y ago

Co-pilot produces original code (as in code that has never been written before). It's not just combining snippets. This should surprise no one who has seen the evolution of language models. Take a look at Kapathy's great write up from way back in 2015[1]. This generates Wikipedia syntax from a character-based RNN. It's operating on a per-character basis, and it doesn't have sufficient capacity to memorise large snippets. (The Paul Graham example spells this out: 1M characters in the dataset = 8M bits, and the network has 3.5M parameters). Semantic arguments about "is this intelligence?" I'll let others fight. [1] http://karpathy.github.io/2015/05/21/rnn-effectiveness/ http://karpathy.github.io/2015/05/21/rnn-effectiveness/

refulgentis 5y ago
You're swapping "It does, at least occasionally, combine snippets" for "it's just combining snippets"
- nl 5y ago
  That is can combine snippets doesn't mean the OP's understanding of how the system works is correct. They seem to believe it is a database system. That's really not how this works, and the fact it behaves like one sometimes is disguising what it is doing. If I say "write a "for" loop from 0 to 10 in Python" probably 50% of implementations by Python programmers will look exactly the same. Some will be retrieving that from memory, but many will be using a generative process that generates the same code, because they've seen and done similar things thousands of times before. A neural network is doing a similar thing. "Write quicksort" makes it start generating tokens, and the loss function has optimised it to generate them in an order it has seen before. It's probably seen a decent number of variations of quicksort, so you might get a mix of what it has seen before. For other pieces of code it has only seen one implementation, so it will generate something very similar. There could be local variations (eg, it sees lots of loops, so it might use a different variation) but in general it will be very similar. But this isn't a database lookup function - it's generative against a loss function. This is subtle distinction, but it is reasonable that people on HN understand this.
  - 5y ago