5 ms·

It certainly seems to be a laundering enabler. Say that you want to un-GPL-ify some famous copylefted code that is on the training database. You type a first

It certainly seems to be a laundering enabler. Say that you want to un-GPL-ify some famous copylefted code that is on the training database. You type a first innocuous characters of it, then the co-pilot keeps completing the rest of the same exact code, for it offers a perfect match. If the completion is not exact, you "twiddle" it a bit until it becomes. Bang! you have a non-gpl copy of the program! Moreover, it is 100% yours and you can re-license it as you want. This will be a boon for copyleft-allergic developers!

rlpb 5y ago
> Bang! you have a non-gpl copy of the program! Moreover, it is 100% yours and you can re-license it as you want. This will be a boon for copyleft-allergic developers! Thinking that this would conveniently bypass the fact that your goal was to copy the code seems to be the most common legal fallacy amongst software developers. The law will see straight through you, and you will be found to have infringed copyright. The reason is well explained in "What Colour are your bits?" (https://ansuz.sooke.bc.ca/entry/23 https://ansuz.sooke.bc.ca/entry/23).
- enriquto 5y ago
  My message was sarcastic. I'm worried about accidental conversion of free software into proprietary. I mean, "accidental" locally, in each particular instance; but maybe non accidental in the grand scheme of things. EDIT: to I can write my worry, semi-jokingly, as a conspiracy theory: Microsoft is using thousands of unsuspecting (and unwilling) developers to turn a huge copylefted corpus of algorithms into non-copylefted implementations. Even assuming that developers that use the co-pilot use non-copyleft licenses only 50% of the time, there's still a constant trickling of un-copyleftization.
freshhawk 5y ago
I suppose someone should make a OS-generating AI, conceptually it can just have windows, osx and some linux distros in it and output one based on a question about favorite color or something. You'd just have to wrap it in a nice complex model representation so it's a black box you fed example OS's with some meta-data into and it happens to output this very useful data. After all, once you use something as input to a machine learning model apparently the license disappears. Sweet.
- bogwog 5y ago
  That would be interesting: * Someone leaks Windows 10/11 source code * Copilot picks it up in its training data * Someone uses copilot to generate a Windows clone and starts selling it I wonder how Microsoft would react to that. I wonder if they've manually blacklisted leaked source code from Windows (or other Microsoft products) so that it doesn't show up in Copilot's training data. If they have, that means Microsoft recognizes the IP risks of having your code in that data set, which would make this Copilot thing not just the result of poor planning/maybe a little incompetence, but something much more devious and malicious. If Microsoft is going to defend this project, they should introduce all of their own source code into the training data.
  - aj3 5y ago
    Come on, there is a huge gap between 1) writing a single function (potentially incorrectly) with a known prototype/interface and a description and 2) designing interfaces, datatypes and APIs themselves.
    - bogwog 5y ago
      Why would you need to design anything? Just copy official Windows headers and use copilot to implement individual functions. Maybe if the signature matches perfectly, copilot will even pull in the exact implementation from the Windows source code.
  - treesprite82 5y ago
    > Someone uses copilot to generate a Windows clone You could test this with one of Microsoft's products that is already on GitHub - like VSCode. I doubt you would get anywhere with just copilot.
    - bogwog 5y ago
      You probably won't get an entire operating system out of it, but I could totally see a project like Wine using it to implement missing parts of the Win32 API and improve their existing implementations.
methyl 5y ago
What stops you to do the same, without the AI part?
- petercooper 5y ago
  That's what I was wondering. I've never been interested enough to steal anyone else's code, but with all the code transformers and processing tools nowadays, I imagine it's trivial to translate source code into a functionally equivalent but stylistically unique version?
  - pjerem 5y ago
    The question is not if it's trivial or not, but if it is legal or not. You can already technically steal GPLv2 by obfuscating it.
    - formerly_proven 5y ago
      Assuming ML models are causal, then bits of GPL code that fall out of the model have to have the color GPL, because the only way they could've gotten there was to train the ML using GPL-colored bits. It seems to me like the answer here is pretty obvious, it doesn't really matter how you copy a work.
      - Rapzid 5y ago
        Bits?
shadilay 5y ago
Would it be possible to do this in reverse assuming the AI has some proprietary code in its training data?
bogwog 5y ago
Yes this is a concern, but I'm not sure if the AI is actually able to "generate" a non-trivial piece of code. If you tell it to generate "a function for calculating the barycentric coordinates of a ray-triangle intersection", you might get a working implementation of a popular algorithm, adapted to your language and existing class/function/variable names. But if you tell it to generate "a smartphone operating system", it probably won't work...and if it does, it would most likely use giant chunks of Android's codebase. And if that's true, it means that copilot isn't really generating anything. It's just a (high-tech) search engine that knows how to adapt the code it finds to fit your codebase. That's still a really cool technology and worth exploring, but it doesn't do enough to justify ignoring software licenses.
- treis 5y ago
  >But if you tell it to generate "a smartphone operating system", it probably won't work...and if it does, it would most likely use giant chunks of Android's codebase. But since now APIs are unprotected you could feed it all of the class structure and method signatures to have it fill in the blanks. I don't know if that gets you a working operating system but it seems like it will get you quite a long way
taneq 5y ago
1) Type a comment like // The following code implements the functionality of <popular GPL'd library> 2) Have library implemented magically for you 3) Delete top comment if necessary :P (It's pretty unlikely that this will actually work but the approach could well do.)
saba2008 5y ago
How is it different from just copy-pasting? It does add some degree of plausible deniability (accidental violation, instead of intentional), but I don't think it would matter much.
alkonaut 5y ago
I don’t think most of us are scared enough of being “tainted” by the sight of a GPL snippet that we’d bother. Besides, if you want to target a specific snippet so you can type the start to prime the recognition - you already saw it? Why not just copy it and then edit it? If a snippet is changed both logically and syntactically to not resemble the original, then it’s no longer the original and you aren’t in any licensing trouble. There is no meaningful difference between that manual washing and a clean room implementation. All the ML changes here is the accidental vs deliberate. But it will be a worse wash than your manual one.

rlpb 5y ago

> Bang! you have a non-gpl copy of the program! Moreover, it is 100% yours and you can re-license it as you want. This will be a boon for copyleft-allergic developers! Thinking that this would conveniently bypass the fact that your goal was to copy the code seems to be the most common legal fallacy amongst software developers. The law will see straight through you, and you will be found to have infringed copyright. The reason is well explained in "What Colour are your bits?" (https://ansuz.sooke.bc.ca/entry/23 https://ansuz.sooke.bc.ca/entry/23).

enriquto 5y ago
My message was sarcastic. I'm worried about accidental conversion of free software into proprietary. I mean, "accidental" locally, in each particular instance; but maybe non accidental in the grand scheme of things. EDIT: to I can write my worry, semi-jokingly, as a conspiracy theory: Microsoft is using thousands of unsuspecting (and unwilling) developers to turn a huge copylefted corpus of algorithms into non-copylefted implementations. Even assuming that developers that use the co-pilot use non-copyleft licenses only 50% of the time, there's still a constant trickling of un-copyleftization.

freshhawk 5y ago

I suppose someone should make a OS-generating AI, conceptually it can just have windows, osx and some linux distros in it and output one based on a question about favorite color or something. You'd just have to wrap it in a nice complex model representation so it's a black box you fed example OS's with some meta-data into and it happens to output this very useful data. After all, once you use something as input to a machine learning model apparently the license disappears. Sweet.

bogwog 5y ago
That would be interesting: * Someone leaks Windows 10/11 source code * Copilot picks it up in its training data * Someone uses copilot to generate a Windows clone and starts selling it I wonder how Microsoft would react to that. I wonder if they've manually blacklisted leaked source code from Windows (or other Microsoft products) so that it doesn't show up in Copilot's training data. If they have, that means Microsoft recognizes the IP risks of having your code in that data set, which would make this Copilot thing not just the result of poor planning/maybe a little incompetence, but something much more devious and malicious. If Microsoft is going to defend this project, they should introduce all of their own source code into the training data.
- aj3 5y ago
  Come on, there is a huge gap between 1) writing a single function (potentially incorrectly) with a known prototype/interface and a description and 2) designing interfaces, datatypes and APIs themselves.
  - bogwog 5y ago
    Why would you need to design anything? Just copy official Windows headers and use copilot to implement individual functions. Maybe if the signature matches perfectly, copilot will even pull in the exact implementation from the Windows source code.
- treesprite82 5y ago
  > Someone uses copilot to generate a Windows clone You could test this with one of Microsoft's products that is already on GitHub - like VSCode. I doubt you would get anywhere with just copilot.
  - bogwog 5y ago
    You probably won't get an entire operating system out of it, but I could totally see a project like Wine using it to implement missing parts of the Win32 API and improve their existing implementations.

methyl 5y ago

What stops you to do the same, without the AI part?

petercooper 5y ago
That's what I was wondering. I've never been interested enough to steal anyone else's code, but with all the code transformers and processing tools nowadays, I imagine it's trivial to translate source code into a functionally equivalent but stylistically unique version?
- pjerem 5y ago
  The question is not if it's trivial or not, but if it is legal or not. You can already technically steal GPLv2 by obfuscating it.
  - formerly_proven 5y ago
    Assuming ML models are causal, then bits of GPL code that fall out of the model have to have the color GPL, because the only way they could've gotten there was to train the ML using GPL-colored bits. It seems to me like the answer here is pretty obvious, it doesn't really matter how you copy a work.
    - Rapzid 5y ago
      Bits?

shadilay 5y ago

Would it be possible to do this in reverse assuming the AI has some proprietary code in its training data?

bogwog 5y ago

Yes this is a concern, but I'm not sure if the AI is actually able to "generate" a non-trivial piece of code. If you tell it to generate "a function for calculating the barycentric coordinates of a ray-triangle intersection", you might get a working implementation of a popular algorithm, adapted to your language and existing class/function/variable names. But if you tell it to generate "a smartphone operating system", it probably won't work...and if it does, it would most likely use giant chunks of Android's codebase. And if that's true, it means that copilot isn't really generating anything. It's just a (high-tech) search engine that knows how to adapt the code it finds to fit your codebase. That's still a really cool technology and worth exploring, but it doesn't do enough to justify ignoring software licenses.

treis 5y ago
>But if you tell it to generate "a smartphone operating system", it probably won't work...and if it does, it would most likely use giant chunks of Android's codebase. But since now APIs are unprotected you could feed it all of the class structure and method signatures to have it fill in the blanks. I don't know if that gets you a working operating system but it seems like it will get you quite a long way

taneq 5y ago

1) Type a comment like // The following code implements the functionality of <popular GPL'd library> 2) Have library implemented magically for you 3) Delete top comment if necessary :P (It's pretty unlikely that this will actually work but the approach could well do.)

saba2008 5y ago

How is it different from just copy-pasting? It does add some degree of plausible deniability (accidental violation, instead of intentional), but I don't think it would matter much.

alkonaut 5y ago

I don’t think most of us are scared enough of being “tainted” by the sight of a GPL snippet that we’d bother. Besides, if you want to target a specific snippet so you can type the start to prime the recognition - you already saw it? Why not just copy it and then edit it? If a snippet is changed both logically and syntactically to not resemble the original, then it’s no longer the original and you aren’t in any licensing trouble. There is no meaningful difference between that manual washing and a clean room implementation. All the ML changes here is the accidental vs deliberate. But it will be a worse wash than your manual one.