4 ms·

Shouldn't I own part of gpt3 because it's trained on my data and likely spitting out what I have commented somewhere on the internet? I think ml models should

Shouldn't I own part of gpt3 because it's trained on my data and likely spitting out what I have commented somewhere on the internet? I think ml models should be public unless the data itself isn't.

searchableguy 6y ago
Curious about the downvotes. If you remove compute and code, what remains is data that isn't owned by a single entity in the case of gpt3. You can sell both compute and code which is what you own but should you be able to sell data? You can't translate a commerical book without paying the copyright holder. You own the copyright for the translation but everything else still remains that of the original author. Why wouldn't this apply in case of a machine learning model trained on my images, messages, intents, likeness, etc?
- isx726552 6y ago
  But you don’t understand, OpenAI has hype and “rockstars” and big names attached, so those rules don’t apply. Your data? It’s their data now. (And Microsoft’s.)
- PeterisP 6y ago
  The current legal understanding is that according to the current copyright laws the ML models trained on some data are not considered derivative works of that data, so if the data was obtained legally and without other restrictions (e.g. if the researchers sign a contract with the data owner that gives them access to data but agree to certain conditions) then the authors of the data do not have any claim on some rights to the resulting model. One aspect of this is that historically statistical models calculated from large volumes of text (which is a notion that predates computers, e.g. frequency dictionaries and the whole [sub]field of quantitative corpus linguistics) have been considered facts about that corpus of text and thus not copyrightable at all or (depending on jurisdiction) entitled to different set of protections/limitations assigned to compilations of facts, which give some rights to the people who compiled the facts but no rights to the source of these facts (since facts as such can't be copyrighted). This also applies to many forms of analysis of audiovisual data, where the copyrights of the source works do not transfer to the results of the statistical or qualitative analysis and can't limit their creation, distribution or sale. The appropriate analogy to a commercial book or movie is not a translation, but some analysis of it - e.g. a thorough literary review and critique of some book or movie is a separate work with its own copyright, and the original author has no claim on it despite the fact that is (obviously) based on the contents of the work and describes it in great detail. Including verbatim fragments of the work is limited (fair use allows some inclusions but not all), but all the other details are not. Also, the whole notion of copyrightability of ML model weight files is interesting and IMHO not settled. You could argue that there is some creative expression in forming the model (which would support it being copyrightable) or you could argue that it's a mechanistic result of the application of some algorithm and settings (which have the creative part, and are copyrightable on their own), and so the output can't be copyrightable, no matter how much work (human or machine), time and cost it took - at least in USA copyright law doctrine (e.g. Feist Publications v. Rural Telephone Service) is that mere "sweat of the brow" (no matter how much) does not entitle a work to copyright protection; it requires application of human creativity to create an original work, and automated processes can't satisfy that requirement. So from the perspective of GPT-3 authors it might be prudent to distribute the model only to people who agree to specific, explicit contractual/licensing restrictions on redistribution and usage of the model instead of simply relying on the default application of copyright laws. And crucially, if some output is not copyrightable, it can't be considered a derived work to which the authors of a source work could have some claim.
  - searchableguy 6y ago
    Thanks for your answer. I appreciate it.
FeepingCreature 6y ago
Should you own part of my brain if I read your comment?
- divtiwari 6y ago
  Good one !!!
- searchableguy 6y ago
  No. As per my layman understanding of copyright, I will have to prove at least: 1. Artistic/creative expression in my comments. 2. Direct commerical interest. If you are not benefiting from my comment directly, it can be considered fair use and hard to prove otherwise. In gpt3's case, it outputs artistic expression in the form of poetry/stories and direct commerical interest from data used by common crawl and their api service. You can argue about derivatives and so on. It's not clear cut in any way but your analogy isn't a good one. We can also give some leeway to humans compared to machines as latter is more deterministic. We know that gpt 3 was trained on which data. We can't say that for sure about you.
  - FeepingCreature 6y ago
    > In gpt3's case, it outputs artistic expression in the form of poetry/stories and direct commerical interest from data used by common crawl and their api service. Okay, so you don't own part of my brain, but StackOverflow would? Because I definitely commercially use information I've read there. And I mean, I know English because of all the English material I've read. If I write poetry, I'm working off training data I've read online. It would be borderline impossible to be an artist under these rules. > We know that gpt 3 was trained on which data. We don't know what data GPT uses for a given answer. We don't know what data I use for a given answer. But we know what data I read, and we know what data GPT read. Far as I can tell, the brain analogy is 1:1 here.
    - searchableguy 6y ago
      > Okay, so you don't own part of my brain, but StackOverflow would? Because I definitely commercially use information I've read there. They have a well established license you need to follow to use the answers posted on their site and what freedom it provides you. Using stackoverflow answer to add little bit of code in your codebase is different than reselling stackoverflow answers behind a pay wall. > But we know what data I read, and we know what data GPT read No unless you are under surveillance 24/7. And as I already said, machines are different than people. Copyright laws acknowledge that so human situation wouldn't be comparable. You aren't mass producing creativity and selling it.
natch 6y ago
It makes its own new arrangements of words. It’s not just clips of what it was trained with pasted together.
- searchableguy 6y ago
  That's why I gave an example of translation. There are new sequences as well as words but the original owner still own the copyright based on the observed meaning or similarities. If you try searching some of the outputs from gpt, you will come across original source in many cases which are slightly or moderately different.
  - natch 6y ago
    This is an area open to interpretation but I wouldn’t go so far as to say someone who used a phrase before owns the copyright to a new work that happens to contain that phrase. But yes they certainly own the copyright to their original entire work in its original form. If you’re talking about excerpts longer than a phrase it does become more murky for sure.

Shouldn't I own part of gpt3 because it's trained on my data and likely spitting out what I have commented somewhere on the internet? I think ml models should be public unless the data itself isn't.

searchableguy 6y ago

Curious about the downvotes. If you remove compute and code, what remains is data that isn't owned by a single entity in the case of gpt3. You can sell both compute and code which is what you own but should you be able to sell data? You can't translate a commerical book without paying the copyright holder. You own the copyright for the translation but everything else still remains that of the original author. Why wouldn't this apply in case of a machine learning model trained on my images, messages, intents, likeness, etc?

isx726552 6y ago
But you don’t understand, OpenAI has hype and “rockstars” and big names attached, so those rules don’t apply. Your data? It’s their data now. (And Microsoft’s.)
PeterisP 6y ago
The current legal understanding is that according to the current copyright laws the ML models trained on some data are not considered derivative works of that data, so if the data was obtained legally and without other restrictions (e.g. if the researchers sign a contract with the data owner that gives them access to data but agree to certain conditions) then the authors of the data do not have any claim on some rights to the resulting model. One aspect of this is that historically statistical models calculated from large volumes of text (which is a notion that predates computers, e.g. frequency dictionaries and the whole [sub]field of quantitative corpus linguistics) have been considered facts about that corpus of text and thus not copyrightable at all or (depending on jurisdiction) entitled to different set of protections/limitations assigned to compilations of facts, which give some rights to the people who compiled the facts but no rights to the source of these facts (since facts as such can't be copyrighted). This also applies to many forms of analysis of audiovisual data, where the copyrights of the source works do not transfer to the results of the statistical or qualitative analysis and can't limit their creation, distribution or sale. The appropriate analogy to a commercial book or movie is not a translation, but some analysis of it - e.g. a thorough literary review and critique of some book or movie is a separate work with its own copyright, and the original author has no claim on it despite the fact that is (obviously) based on the contents of the work and describes it in great detail. Including verbatim fragments of the work is limited (fair use allows some inclusions but not all), but all the other details are not. Also, the whole notion of copyrightability of ML model weight files is interesting and IMHO not settled. You could argue that there is some creative expression in forming the model (which would support it being copyrightable) or you could argue that it's a mechanistic result of the application of some algorithm and settings (which have the creative part, and are copyrightable on their own), and so the output can't be copyrightable, no matter how much work (human or machine), time and cost it took - at least in USA copyright law doctrine (e.g. Feist Publications v. Rural Telephone Service) is that mere "sweat of the brow" (no matter how much) does not entitle a work to copyright protection; it requires application of human creativity to create an original work, and automated processes can't satisfy that requirement. So from the perspective of GPT-3 authors it might be prudent to distribute the model only to people who agree to specific, explicit contractual/licensing restrictions on redistribution and usage of the model instead of simply relying on the default application of copyright laws. And crucially, if some output is not copyrightable, it can't be considered a derived work to which the authors of a source work could have some claim.
- searchableguy 6y ago
  Thanks for your answer. I appreciate it.

FeepingCreature 6y ago

Should you own part of my brain if I read your comment?

divtiwari 6y ago
Good one !!!
searchableguy 6y ago
No. As per my layman understanding of copyright, I will have to prove at least: 1. Artistic/creative expression in my comments. 2. Direct commerical interest. If you are not benefiting from my comment directly, it can be considered fair use and hard to prove otherwise. In gpt3's case, it outputs artistic expression in the form of poetry/stories and direct commerical interest from data used by common crawl and their api service. You can argue about derivatives and so on. It's not clear cut in any way but your analogy isn't a good one. We can also give some leeway to humans compared to machines as latter is more deterministic. We know that gpt 3 was trained on which data. We can't say that for sure about you.
- FeepingCreature 6y ago
  > In gpt3's case, it outputs artistic expression in the form of poetry/stories and direct commerical interest from data used by common crawl and their api service. Okay, so you don't own part of my brain, but StackOverflow would? Because I definitely commercially use information I've read there. And I mean, I know English because of all the English material I've read. If I write poetry, I'm working off training data I've read online. It would be borderline impossible to be an artist under these rules. > We know that gpt 3 was trained on which data. We don't know what data GPT uses for a given answer. We don't know what data I use for a given answer. But we know what data I read, and we know what data GPT read. Far as I can tell, the brain analogy is 1:1 here.
  - searchableguy 6y ago
    > Okay, so you don't own part of my brain, but StackOverflow would? Because I definitely commercially use information I've read there. They have a well established license you need to follow to use the answers posted on their site and what freedom it provides you. Using stackoverflow answer to add little bit of code in your codebase is different than reselling stackoverflow answers behind a pay wall. > But we know what data I read, and we know what data GPT read No unless you are under surveillance 24/7. And as I already said, machines are different than people. Copyright laws acknowledge that so human situation wouldn't be comparable. You aren't mass producing creativity and selling it.

natch 6y ago

It makes its own new arrangements of words. It’s not just clips of what it was trained with pasted together.

searchableguy 6y ago
That's why I gave an example of translation. There are new sequences as well as words but the original owner still own the copyright based on the observed meaning or similarities. If you try searching some of the outputs from gpt, you will come across original source in many cases which are slightly or moderately different.
- natch 6y ago
  This is an area open to interpretation but I wouldn’t go so far as to say someone who used a phrase before owns the copyright to a new work that happens to contain that phrase. But yes they certainly own the copyright to their original entire work in its original form. If you’re talking about excerpts longer than a phrase it does become more murky for sure.