6 ms·

The analogy is wrong, AI does not at all look similar to a human mind, it is a more complex algorithm , a obfuscated script that if done wrong will output the

The analogy is wrong, AI does not at all look similar to a human mind, it is a more complex algorithm , a obfuscated script that if done wrong will output the exact input you used it for training. This can happen probably very often withy original stuff and less often with trivial stuff.

ghaff 4y ago
>AI does not at all look similar to a human mind That is a matter of some debate among neurophysiologists and others. There are clearly differences but there also seem to be similarities that go beyond mere analogy.
- simion314 4y ago
  That depends on how you define similarities. This ANN AIs are a complex version of a polynomial interpolation , you have some points from a measurements and you build a function to fit those points, so in the end you have a function with a giant matrix of data as a "knowledge" database behind. But even with the human mind, I am 100% sure Microsoft would not like it if I read their precious Windows source code and then go and work on wine, they might claim that I did not learned from their code but memorize secret algorithms , or maybe would claim that I did not ask for permissions first.
  - ghaff 4y ago
    These days I doubt Microsoft would care much. But to your broader point, yes, that's why you do clean room implementations of public APIs. If you've never seen an implementation, you can't really be accused of copying it. This is really different though in that this code is all open source. The argument is that sufficient code is being verbatim copies so as to be a derivative work.
    - simion314 4y ago
      >This is really different though in that this code is all open source. Open source or proprietary you still have to respect the license. >The argument is that sufficient code is being verbatim copies so as to be a derivative work. Yeah, it also depends what derivative work means, I could make a script that inputs MS code and outputs it with obfuscation, is it derivative work or will MS complain that is not enough and I did not had permission to do this, AFAIK code on GitHub does not have the permssions to be used as input for copilot, and not all code there has the permissions from all authors to be hosted there. What MS should do is 1 use their proprietary repos as input too to show us that they think the results are derivative code 2 use their customer private code too to prove again that they are ready to prove in court that this is derivative work 3 open the model with no restrictions, then we could say MS is doing it for the advancement of society and sure charge for the service 4 promise to defend any developer in case their AI outputs copyrighted content.
      - ghaff 4y ago
        >Open source or proprietary you still have to respect the license. >they might claim that I did not learned from their code but memorize secret algorithms The difference is that you can't claim open source code is a trade secret that contains algorithms that you can't reimplement in your own code.
        simion314 4y ago
        >The difference is that you can't claim open source code is a trade secret that contains algorithms that you can't reimplement in your own code. You can't use "trade secret" but MS Windows code is also no longer secret, they shared the code with third parties and was also leaked, so they can claim you do not respect the license or TOS,NDAs etc. Not sure if "secret" has anything to do with this, proprietary stuff is not secret, it could be source availleble or be a scripting language stuff or something trivial to find the original implementation like C# and Java
        tuckerman 4y ago
        It’s possible for there to be patent encumbered algorithms in open source code though. A license like Apache will contain a free license to use them to use/distribute the open source work… which I’m not sure would transfer if copilot happens to plop some unlicensed, unattributed violating code in your code base.
  - ntoskrnl 4y ago
    The leaked Windows source is actually hosted right on GitHub. I wouldn't be surprised if it was part of Copilot's training set.
cdrini 4y ago
AI is not similar to a human mind in the general case (ie general AI), but in the context of reading and learning and generating code there are similarities. And I would argue that humans are also very susceptible to "output the exact input used for training".
- simion314 4y ago
  How is similar in generating code? From my case I don't create code by combining previous seen code, for example I can write a Lua/Haskell script now even in my memory I have no Lua/Haskell code stored. I can do it because I create a model of the problem, then I create structures of data and operations on those data, only the final step is to look up the syntax and standard libraries to generate the code. Can someone prove that it at least managed to understand trivial algorithms, like this is a find.sort,reverse,filter,map operation and it can say map an algorithm from one language to other? Thinking at it , bad students when learning programming are doing this kind of stuff, they start writing stuff from memory that looks like valid code , one student wrote something like if(int i =0; i < n; i++) , clearly this student did not understand the lesson so Copilot will do a similar mistake just one level higher
  - cdrini 4y ago
    I definitely agree copilot isn't understanding the problem space of your code. But I also don't believe it's simply remixing code samples from its training set. It's somewhere in between. I don't know the internals, but it looks to me like it's operating at a few layers of abstraction over literal code and syntax. It find patterns in the relationships between symbols and references. Not to mention since it's GPT-3 based, it's cross referencing the "meaning" in these abstract relationships with the meaning of plain English text written in comments, too. These pieces are similar to humans. Just like a human, copilot doesn't have a ton of exact literal code in its model. It's seen lots of code and has patterns and relationships in its model. That's why just like a human, it can translate ideas between different coding languages -- my guess would be it can write an algorithm its seen in C in F# even if its never seen that algorithm in F#; but that's hard to prove. Just like a human though, it might have some literal snippets, though. It can definitely translate between languages ; I've done that a few times. (It can even translate human languages eg English to French! I've done that sometimes for fun) I would highly recommend giving the trial version a go, if only to better understand how it works. Whether it "understands" eg a map operation... That's hard to prove. Can you think of any experiments? I think it understand the relationship between the English word map and the code patterns often associated with that word. I think your example is close. Copilot is a lot like an inexperienced developer. It doesn't (usually) make syntax errors, but because it doesn't understand your problem space, and because it doesn't have as many layers of abstraction as a human does, it does sometimes make silly mistakes. I definitely wouldn't trust it to write an entire program on its own! But with a human in the loop doing the more complicated abstract pieces of coding, it handles the more simple menial pieces pretty well!

ghaff 4y ago

>AI does not at all look similar to a human mind That is a matter of some debate among neurophysiologists and others. There are clearly differences but there also seem to be similarities that go beyond mere analogy.

simion314 4y ago
That depends on how you define similarities. This ANN AIs are a complex version of a polynomial interpolation , you have some points from a measurements and you build a function to fit those points, so in the end you have a function with a giant matrix of data as a "knowledge" database behind. But even with the human mind, I am 100% sure Microsoft would not like it if I read their precious Windows source code and then go and work on wine, they might claim that I did not learned from their code but memorize secret algorithms , or maybe would claim that I did not ask for permissions first.
- ghaff 4y ago
  These days I doubt Microsoft would care much. But to your broader point, yes, that's why you do clean room implementations of public APIs. If you've never seen an implementation, you can't really be accused of copying it. This is really different though in that this code is all open source. The argument is that sufficient code is being verbatim copies so as to be a derivative work.
  - simion314 4y ago
    >This is really different though in that this code is all open source. Open source or proprietary you still have to respect the license. >The argument is that sufficient code is being verbatim copies so as to be a derivative work. Yeah, it also depends what derivative work means, I could make a script that inputs MS code and outputs it with obfuscation, is it derivative work or will MS complain that is not enough and I did not had permission to do this, AFAIK code on GitHub does not have the permssions to be used as input for copilot, and not all code there has the permissions from all authors to be hosted there. What MS should do is 1 use their proprietary repos as input too to show us that they think the results are derivative code 2 use their customer private code too to prove again that they are ready to prove in court that this is derivative work 3 open the model with no restrictions, then we could say MS is doing it for the advancement of society and sure charge for the service 4 promise to defend any developer in case their AI outputs copyrighted content.
    - ghaff 4y ago
      >Open source or proprietary you still have to respect the license. >they might claim that I did not learned from their code but memorize secret algorithms The difference is that you can't claim open source code is a trade secret that contains algorithms that you can't reimplement in your own code.
      - simion314 4y ago
        >The difference is that you can't claim open source code is a trade secret that contains algorithms that you can't reimplement in your own code. You can't use "trade secret" but MS Windows code is also no longer secret, they shared the code with third parties and was also leaked, so they can claim you do not respect the license or TOS,NDAs etc. Not sure if "secret" has anything to do with this, proprietary stuff is not secret, it could be source availleble or be a scripting language stuff or something trivial to find the original implementation like C# and Java
      - tuckerman 4y ago
        It’s possible for there to be patent encumbered algorithms in open source code though. A license like Apache will contain a free license to use them to use/distribute the open source work… which I’m not sure would transfer if copilot happens to plop some unlicensed, unattributed violating code in your code base.
- ntoskrnl 4y ago
  The leaked Windows source is actually hosted right on GitHub. I wouldn't be surprised if it was part of Copilot's training set.

cdrini 4y ago

AI is not similar to a human mind in the general case (ie general AI), but in the context of reading and learning and generating code there are similarities. And I would argue that humans are also very susceptible to "output the exact input used for training".

simion314 4y ago
How is similar in generating code? From my case I don't create code by combining previous seen code, for example I can write a Lua/Haskell script now even in my memory I have no Lua/Haskell code stored. I can do it because I create a model of the problem, then I create structures of data and operations on those data, only the final step is to look up the syntax and standard libraries to generate the code. Can someone prove that it at least managed to understand trivial algorithms, like this is a find.sort,reverse,filter,map operation and it can say map an algorithm from one language to other? Thinking at it , bad students when learning programming are doing this kind of stuff, they start writing stuff from memory that looks like valid code , one student wrote something like if(int i =0; i < n; i++) , clearly this student did not understand the lesson so Copilot will do a similar mistake just one level higher
- cdrini 4y ago
  I definitely agree copilot isn't understanding the problem space of your code. But I also don't believe it's simply remixing code samples from its training set. It's somewhere in between. I don't know the internals, but it looks to me like it's operating at a few layers of abstraction over literal code and syntax. It find patterns in the relationships between symbols and references. Not to mention since it's GPT-3 based, it's cross referencing the "meaning" in these abstract relationships with the meaning of plain English text written in comments, too. These pieces are similar to humans. Just like a human, copilot doesn't have a ton of exact literal code in its model. It's seen lots of code and has patterns and relationships in its model. That's why just like a human, it can translate ideas between different coding languages -- my guess would be it can write an algorithm its seen in C in F# even if its never seen that algorithm in F#; but that's hard to prove. Just like a human though, it might have some literal snippets, though. It can definitely translate between languages ; I've done that a few times. (It can even translate human languages eg English to French! I've done that sometimes for fun) I would highly recommend giving the trial version a go, if only to better understand how it works. Whether it "understands" eg a map operation... That's hard to prove. Can you think of any experiments? I think it understand the relationship between the English word map and the code patterns often associated with that word. I think your example is close. Copilot is a lot like an inexperienced developer. It doesn't (usually) make syntax errors, but because it doesn't understand your problem space, and because it doesn't have as many layers of abstraction as a human does, it does sometimes make silly mistakes. I definitely wouldn't trust it to write an entire program on its own! But with a human in the loop doing the more complicated abstract pieces of coding, it handles the more simple menial pieces pretty well!