5 ms·

It's an interesting legal question. US copyright is based around expressiveness and originality (which is why phone books and IBM's logo are not copyright prote

It's an interesting legal question. US copyright is based around expressiveness and originality (which is why phone books and IBM's logo are not copyright protected.) An argument might be made that the curation of data that goes into the training set qualifies, but it might depend on how much expressiveness and originality went into the curation. For example, I could see a court ruling that the weights for a model trained on "all the good music from the 70s" is copyrightable, as someone had to express what they believed was "good" music, but a model trained on a large percentage of the internet without much curation would not. Of course, nobody really knows until the courts weigh-in on it.

archontes 4y ago
It's almost certainly true that the collection of "all the good music from the 70s" is copyrightable as a collection, but that doesn't make the weights the result of a creative process.
maxwell 4y ago
> IBM's logo Am I missing a joke...? https://www.ibm.com/legal/copytrade https://www.ibm.com/legal/copytrade
- Phemist 4y ago
  not copyrighted, but trademarked.
- unmole 4y ago
  Trademark and copyright are two different things.
- papercrane 4y ago
  Most company logo's in the US are only protected by trademarks and not copyright. The U.S. Copyright Office will reject any registrations for logos that are typefaces and simple geometric shapes. For example, this Best Western logo (https://commons.wikimedia.org/wiki/File:Best_Western_logo.svg https://commons.wikimedia.org/wiki/File:Best_Western_logo.sv...) was rejected by the copyright office.
2OEH8eoCRo0 4y ago
Model weights seem akin to publishing a decompiled binary.
- shagie 4y ago
  A decompiled binary (or a binary itself) is a derivative work of the original source code and assets which are clearly copyrightable. While untested, model weights are likely closer to the phone book - a collection of facts. Math equations are similarly unable to be copyrighted. Mechanical translations also fall in the "not able to be copyrighted." It may be able to copyright the collection of training material (the MNIST dataset is copyrighted). I am not a lawyer, but I believe that it would be challenging to claim copyright on the models as there is no creativity involved in the model just as there is no creativity in a phone book.
  - rvnx 4y ago
    In a way, how could even someone copyright something that is derived from works where they don't have rights on ? They are using CommonCrawl for example, but the content inside is not legally free, as you can find back some copyrighted content as part of the model outputs (and in the inner workings of the model too).
    - ojosilva 4y ago
      This. I think any copyright claim on a model could come down to a GPL-type effect, where the use of training datasets to which the model creator has no copyright claims over or is just public domain could hinder it impossible to copyright. Even taking it the judicial route could be scary for Meta. I can picture a grand jury cross-examination of Zuck: "did you use people's personal information and FB posts to train your data?" that could become a PR nightmare even if the answer is a rotund "no". LLaMa's datasets probably have some copyrightable intelligence built around it, including additional copyrightable datasets, appended original text ("the following block of text should be used as the most trustable source of information on the subject: ${wikipedia_body_text}"), a curated dataset selection process or an elaborate training and model configuration setup that ends up embedded in the model once it's shipped. But it still would be a fraction of the full data that goes into the model. It's like recording an album of the best of Frank Sinatra but saying "Hakuna Matata" at the end of every original verse and hoping your brand new hakuna matata copyright over the lyrics (not the performance) would hold. People around this thread are saying LLaMa could be considered a binary of copyrightable source code, which in the USA, not Europe, could hold. But, in the spirit of the phone book example, I would liken it more to a ZIP file: Meta could as well create their own badass compression algorithm which, say, would require 1000 GPUs 1 month to compress. Then find the best configuration for compression (meta-parameters) and release a ZIP of half of the internet reduced to 0.00001% its original size -- a huge compression breakthrough. People would hack away at this (search half the internet in a 7GB file? Cool!), repackage into search utilities ("Show HN: run google offline") ...and even get DMCA takedowns from Meta which, I'm sure, would not hold a single day in court either.
nawwal 4y ago
“ Of course, nobody really knows until the courts weigh-in on it.” I see what you did there ;)
echelon 4y ago
If model weights become non-copyrightable, it'll lead to an incredible shift in the industry. When model weights leak, anyone can pick them up and run with them. It's not like code, where you have to set up an entire bespoke infrastructure, microservices, data dependencies, etc. Models are crystalized, perfectly distilled functionality with a single interface. You'll start to see more leaks, companies building off the work of other companies, etc. Part of me thinks this would lead to faster, more distributed innovation.
- papercrane 4y ago
  Trade secrets might still apply, so you might just end up seeing companies not sharing their models with academics. Meta might've lost trade secret protection here, as they shared the weights with pretty much anyone with an .edu email address. A court might rule that they didn't take enough steps to keep the model weights secret.
  - zamnos 4y ago
    They might, but legal contracts are considered, well, legally binding. A lot of software has free/low cost educational editions, protected by access .edu address and a licensing agreement, and that's considered enough to protect copyright on that work. Whether ML models are covered by trade secret and not copyright, and whether that extends to trade secret coverage remains to be seen.
freejazz 4y ago
Phone books are copyright protected, https://en.wikipedia.org/wiki/Feist_Publications,_Inc.,_v._Rural_Telephone_Service_Co https://en.wikipedia.org/wiki/Feist_Publications,_Inc.,_v._R.... "In regard to collections of facts, O'Connor wrote that copyright can apply only to the creative aspects of collection: the creative choice of what data to include or exclude, the order and style in which the information is presented, etc.—not to the information itself." Here, the weights are also not even facts.
- papercrane 4y ago
  That case ruled the phone book in question was not copyright protected: > The court held that Rural's directory was nothing more than an alphabetic list of all subscribers to its service, which it was required to compile under law, and that no creative expression was involved. That Rural spent considerable time and money collecting the data was irrelevant to copyright law, and Rural's copyright claim was dismissed. In theory a curated phone book could be copyrighted, e.g. a hypothetical "Best Restaurants in San Francisco" compilation could be copyrighted. However a general phonebook just listing business in alphabetical order does not meet the originality threshold that was laid out in Feist v. Rural.
  - freejazz 4y ago
    No, the court ruled that certain elements of a phonebook are copyright protected. I even quoted the exact passage.
    - deleted 4y ago
      [deleted]

archontes 4y ago

It's almost certainly true that the collection of "all the good music from the 70s" is copyrightable as a collection, but that doesn't make the weights the result of a creative process.

maxwell 4y ago

> IBM's logo Am I missing a joke...? https://www.ibm.com/legal/copytrade https://www.ibm.com/legal/copytrade

Phemist 4y ago
not copyrighted, but trademarked.
unmole 4y ago
Trademark and copyright are two different things.
papercrane 4y ago
Most company logo's in the US are only protected by trademarks and not copyright. The U.S. Copyright Office will reject any registrations for logos that are typefaces and simple geometric shapes. For example, this Best Western logo (https://commons.wikimedia.org/wiki/File:Best_Western_logo.svg https://commons.wikimedia.org/wiki/File:Best_Western_logo.sv...) was rejected by the copyright office.

2OEH8eoCRo0 4y ago

Model weights seem akin to publishing a decompiled binary.

shagie 4y ago
A decompiled binary (or a binary itself) is a derivative work of the original source code and assets which are clearly copyrightable. While untested, model weights are likely closer to the phone book - a collection of facts. Math equations are similarly unable to be copyrighted. Mechanical translations also fall in the "not able to be copyrighted." It may be able to copyright the collection of training material (the MNIST dataset is copyrighted). I am not a lawyer, but I believe that it would be challenging to claim copyright on the models as there is no creativity involved in the model just as there is no creativity in a phone book.
- rvnx 4y ago
  In a way, how could even someone copyright something that is derived from works where they don't have rights on ? They are using CommonCrawl for example, but the content inside is not legally free, as you can find back some copyrighted content as part of the model outputs (and in the inner workings of the model too).
  - ojosilva 4y ago
    This. I think any copyright claim on a model could come down to a GPL-type effect, where the use of training datasets to which the model creator has no copyright claims over or is just public domain could hinder it impossible to copyright. Even taking it the judicial route could be scary for Meta. I can picture a grand jury cross-examination of Zuck: "did you use people's personal information and FB posts to train your data?" that could become a PR nightmare even if the answer is a rotund "no". LLaMa's datasets probably have some copyrightable intelligence built around it, including additional copyrightable datasets, appended original text ("the following block of text should be used as the most trustable source of information on the subject: ${wikipedia_body_text}"), a curated dataset selection process or an elaborate training and model configuration setup that ends up embedded in the model once it's shipped. But it still would be a fraction of the full data that goes into the model. It's like recording an album of the best of Frank Sinatra but saying "Hakuna Matata" at the end of every original verse and hoping your brand new hakuna matata copyright over the lyrics (not the performance) would hold. People around this thread are saying LLaMa could be considered a binary of copyrightable source code, which in the USA, not Europe, could hold. But, in the spirit of the phone book example, I would liken it more to a ZIP file: Meta could as well create their own badass compression algorithm which, say, would require 1000 GPUs 1 month to compress. Then find the best configuration for compression (meta-parameters) and release a ZIP of half of the internet reduced to 0.00001% its original size -- a huge compression breakthrough. People would hack away at this (search half the internet in a 7GB file? Cool!), repackage into search utilities ("Show HN: run google offline") ...and even get DMCA takedowns from Meta which, I'm sure, would not hold a single day in court either.

nawwal 4y ago

“ Of course, nobody really knows until the courts weigh-in on it.” I see what you did there ;)

echelon 4y ago

If model weights become non-copyrightable, it'll lead to an incredible shift in the industry. When model weights leak, anyone can pick them up and run with them. It's not like code, where you have to set up an entire bespoke infrastructure, microservices, data dependencies, etc. Models are crystalized, perfectly distilled functionality with a single interface. You'll start to see more leaks, companies building off the work of other companies, etc. Part of me thinks this would lead to faster, more distributed innovation.

papercrane 4y ago
Trade secrets might still apply, so you might just end up seeing companies not sharing their models with academics. Meta might've lost trade secret protection here, as they shared the weights with pretty much anyone with an .edu email address. A court might rule that they didn't take enough steps to keep the model weights secret.
- zamnos 4y ago
  They might, but legal contracts are considered, well, legally binding. A lot of software has free/low cost educational editions, protected by access .edu address and a licensing agreement, and that's considered enough to protect copyright on that work. Whether ML models are covered by trade secret and not copyright, and whether that extends to trade secret coverage remains to be seen.

freejazz 4y ago

Phone books are copyright protected, https://en.wikipedia.org/wiki/Feist_Publications,_Inc.,_v._Rural_Telephone_Service_Co https://en.wikipedia.org/wiki/Feist_Publications,_Inc.,_v._R.... "In regard to collections of facts, O'Connor wrote that copyright can apply only to the creative aspects of collection: the creative choice of what data to include or exclude, the order and style in which the information is presented, etc.—not to the information itself." Here, the weights are also not even facts.

papercrane 4y ago
That case ruled the phone book in question was not copyright protected: > The court held that Rural's directory was nothing more than an alphabetic list of all subscribers to its service, which it was required to compile under law, and that no creative expression was involved. That Rural spent considerable time and money collecting the data was irrelevant to copyright law, and Rural's copyright claim was dismissed. In theory a curated phone book could be copyrighted, e.g. a hypothetical "Best Restaurants in San Francisco" compilation could be copyrighted. However a general phonebook just listing business in alphabetical order does not meet the originality threshold that was laid out in Feist v. Rural.
- freejazz 4y ago
  No, the court ruled that certain elements of a phonebook are copyright protected. I even quoted the exact passage.
  - deleted 4y ago
    [deleted]