7 ms·
Is there precedent on model weights being copyrightable in the first place? I suppose the recipients of the DMCA notices are unlikely to be willing to contest i
by unnah 4y ago
Is there precedent on model weights being copyrightable in the first place? I suppose the recipients of the DMCA notices are unlikely to be willing to contest it in court, though.
- bottled_poe 4y agoDoesn’t matter. The weights wouldn’t exist without many tens of millions (at least) in investment. The outcome seems very clear.
- papercrane 4y agoUS copyright law cares about originality and expressiveness, not labour and cost. It can be very expensive to collect and print a list of every business and their phone number in a book, but the result is not copyrightable in the US.
- evandale 4y agoThis is a pretty compelling argument to me that the weights shouldn't be able to be copyrighted. In essence, aren't they a list of numbers you can search through to get a result? Would it makes sense to say a "give me a rowboat on the water" prompt (search) is the same as a "give me the phone number of company XYZ" search in a phonebook? What exactly is a prompt anyway? Can you copyright the assembly that a compiler spits out? Is an AI prompt the same as source code and its output is the assembly a compiler would generate? Does that mean the model is a compiler? I assume a compiler can be copyrighted, so then maybe the weights can be copyrighted? Or would it make more sense that the combination of (weights + prompt + seed + output) is copyrightable? I don't have answers to any of these or know if they're reasonable questions but I'm starting to find this all very fascinating.
- postalrat 4y agoIf you are mining Bitcoin can you copyright a block you discover? You probably payed a lot of money for it.
- pygy_ 4y agoIANAL, but I don't think the DMCA applies. Copyright applies to creative work, and model weights aren't that. It is however my understanding that downloading them can be considered a misappropriation of a trade secret. Folks that would contest the bogus DMCA takedown requests would be liable to a trade secret suit.
- rvnx 4y agoarchive.org may be the legal solution ?
- UncleEntity 4y agoYou can’t just assert “this is a secret” and force people you have no contractual relationship with to comply with your assertion. IMHO, not a lawyer. The person who leaked the weights on BitTorrent is definitely liable for violating whatever restrictions they agreed to to get access to them though.
- blackbear_ 4y agoDon't forget that transformers, the basic architecture of these large language models, were introduced by Google, and many other basic building blocks were introduced by universities and research centers all over the world. So no, the outcome is not very clear, at all.
- nl 4y agoThis is irrelevant.
- papercrane 4y agoIt's an interesting legal question. US copyright is based around expressiveness and originality (which is why phone books and IBM's logo are not copyright protected.) An argument might be made that the curation of data that goes into the training set qualifies, but it might depend on how much expressiveness and originality went into the curation. For example, I could see a court ruling that the weights for a model trained on "all the good music from the 70s" is copyrightable, as someone had to express what they believed was "good" music, but a model trained on a large percentage of the internet without much curation would not. Of course, nobody really knows until the courts weigh-in on it.
- archontes 4y agoIt's almost certainly true that the collection of "all the good music from the 70s" is copyrightable as a collection, but that doesn't make the weights the result of a creative process.
- maxwell 4y ago> IBM's logo Am I missing a joke...? https://www.ibm.com/legal/copytrade https://www.ibm.com/legal/copytrade
- Phemist 4y agonot copyrighted, but trademarked.
- unmole 4y agoTrademark and copyright are two different things.
- papercrane 4y agoMost company logo's in the US are only protected by trademarks and not copyright. The U.S. Copyright Office will reject any registrations for logos that are typefaces and simple geometric shapes. For example, this Best Western logo (https://commons.wikimedia.org/wiki/File:Best_Western_logo.svg https://commons.wikimedia.org/wiki/File:Best_Western_logo.sv...) was rejected by the copyright office.
- sillysaurusx 4y agoIf anyone's willing to fund the legal battle, let me know. (You can DM me on Twitter: https://twitter.com/theshawwn https://twitter.com/theshawwn) I'd be willing to issue a DMCA counterclaim for llama-dl on the grounds that model weights are not copyrightable. If it's worth settling the question in court, then this seems like a good opportunity. I wrote more about this further downthread: https://news.ycombinator.com/item?id=35288415 https://news.ycombinator.com/item?id=35288415
- ISL 4y agoIf Meta has registered the work with the copyright office, the statutory damages in this case, should llama-dl lose, might be quite large. Check in with an attorney before launching a battle with an opponent who has unlimited resources. There are likely to be many similar test cases in the coming year, perhaps more-readily fought.
- freejazz 4y agoIt's not a clear issue of law so atty's fees aren't going to be shifted in a case like this.
- loudmax 4y agoA particularly thorny question if data in the training set is copyrighted. Which I presume much of it is, since I was playing with alpaca.cpp and it answered all kinds of pop culture questions I asked it (to varying degrees of accuracy).
- kmeisthax 4y agoYes, but Meta doesn't own that data. They only own the arrangement of what is and isn't in the training set, insamuch as that training set represents human creativity. The process of training model weights is itself purely mechanical. The closest that they could get would be trade secrecy violations, but that only punishes the original leaker and anyone working in concert with them. I'm not sure if anyone's successfully managed to get an entire BitTorrent swarm to be considered misappropriating trade secrets. Presumably at some point, when the trade secret has been violated, you can obtain it without misappropriating - otherwise, how does that not just become Copyright 2.0?
- feanaro 4y agoIf weights are copyrightable, then models should need to obtain permission before using copyrighted works to train them. You can't have it both ways.
- ISL 4y agoEncyclopedias and journalism are copyrightable even when they include articles about copyrighted works (books, films, etc.).
- cwmma 4y agoyeah but that's because there is human creative work in the write ups of the copyrighted work, a court might liken this more to a phone book which which doesn't add any (human) creative expression on top of it.
- feanaro 4y agoNot to mention the final product of encyclopedic work and journalism does not internalize the original it is describing in its entirety, like a language model does. In some sense, a language model takes all that there is to be taken from a given resource, and incorporates it into the weights.
- microtonal 4y agoIANAL, so this is not legal advise. I consulted a legal expert a few years ago about the status of machine learning models and they said it is really unclear. Apparently, if works are transformed enough that the original is not recognizable anymore it may not violate copyright. It hinges a lot on whether the original work is reproduced, so if you could get an LLM to spit out copyrighted texts unmodified, then it would most likely be copyright violation. But I think that doesn't really happen much in practice. On the other hand, Meta can have copyright over the model through 'copyright in compilation', which protects compiled works, regardless of the copyright of the underlying material. So, I fear that it may be possible to have it both ways. But realistically, I think we'll only know for sure when this is fought out in court. Disclaimer: again I am not a lawyer, so take this with a grain of salt.
- MrsPeaches 4y agoIf it’s a specific list, then I would guess so. By it may be that the numbers themselves aren’t copyrightable. In the same way that a list of ingredients can/can’t be copyrighted.
- contravariant 4y agoThis comment is a list of numbers represented as ASCII and is copyrightable.
- archontes 4y agoOnly if a human wrote it.
- MrsPeaches 4y agoIf it’s a specific list, then I would guess so. But it may be that the numbers themselves aren’t copyrightable. In the same way that a list of ingredients can/can’t be copyrighted.
- ineptech 4y agoCopyright-wise, why would the weights be any different from a video file? At the end of the day, LLaMa's weights and fast_and_furious_11.mp4 are both just strings of binary data that some company made to sell.
- JeremyNT 4y agoIt seems intuitively kind of bonkers that they can ignore copyright when generating weights using stuff on the internet, and then turn around and claim copyright on the resulting artifacts.
- mdanger007 4y agoTech companies want it both ways. 1. They own the rights to any user content they store and transmit. 2. They are shielded by Section 230, and immune to liability for its misuse.
- DesiLurker 4y agoIDK, its more like finding recipes to many great restaurant chains all mushed together by a 5th grader whose uncle stole it from them, on the sidewalk. looks like a grey area to me legally but IANAL.