4 ms·
And hopefully this puts to rest all the painfully bad, often anthropomorphizing, takes about how what the LLMs do isn't copyright infringement. It's simple. If
by jrm4 1y ago
And hopefully this puts to rest all the painfully bad, often anthropomorphizing, takes about how what the LLMs do isn't copyright infringement.
It's simple. If you put the works into the LLM, it can later make immediately identifiable, if imperfect, copies of the work. If you didn't put the work in, it wouldn't be able to do that.
The fact that you can't "see the copy" inside is wildly irrelevant.
- orionsbelt 1y agoSo can humans? I can ask a human to draw Mickey Mouse or Superman, and they can! Or recite a poem. Some humans have much better memories and can do this with a far greater degree of fidelity too, just like an LLM vs an average human. If you ask OpenAI to generate an image of your dog as Superman, it will often start to do so, and then it will realize it is copyrighted, and stop. This seems sensible to me. Isn’t it the ultimate creative result that is copyright infringement, and not merely that a model was trained to understand something very well?
- jrm4 1y agoCopyright infringement is the act of creating/using an copy in an unauthorized way. Remember, we can only target humans. So we're not likely to target your guy; but we ARE likely to target "the guy that definitely fed a complete unauthorized copy of the thing into the LLM."
- regularfry 1y agoI just don't get the legal theory here. If I download harry_potter_goblet_fire.txt off some dodgy site, then let's assume that owner of that site has infringed copyright by distributing it. If I upload it again to some other dodgy site, I would also infringe copyright in a similar same way. But that would be naughty so I'm not going to do that. Let's say instead that I feed it into a bunch of janky pytorch scripts with a bunch of other text files, and out pops a bunch of weights. Twice. The first model I build is a classifier. Its output is binary: is this text about wizards, yes/no. The second model I build is an LLM. Its output is text, and (as in the article) you can get imperfect reproductions of parts of the training file out of it with the right prompts. Now, I upload both those sets of weights to HuggingFace. How many times am I supposed to have infringed copyright? Is it: A) Twice (at least), because the act of doing anything whatsoever with harry_potter_goblet_fire.txt without permission is verboten; B) Once, because only one of the models is capable of reproducing the original (even if only approximately); C) Zero, because neither model is capable of a reproduction that would compete with the original; or D) Zero, because I'm not the distributor of the file, and merely processing it - "format shifted" from the book, if you like - is not problematic in itself. Logically I can see justifications for any of B) (tenuously), C), or D). Obviously publishers would want us to think that A) is right, but based on what? I see a lot of moral outrage, but very little actual argument. That makes me think there's nothing there.
- bodhi 1y agoI’ll preface with IANACL, but you seem to be making a moral argument yourself about A), that it is not reasonable. You have, I assume, a licensed copy of Harry Potter. That license restricts you from doing certain activities, like making (distributing? Lets go with distributing) derived works. Your models are derived works. Thus when you distribute your models, you’re violating the licence terms you “agreed” to when you acquired your copy of Harry Potter. This is no judgement by me about whether that’s reasonable or not, just my understanding of the mechanics.
- perching_aix 1y agoRather than a moral argument, I think they're more just disagreeing that the spirit behind copyright law is being violated in that case. So while yes, there may be a rule in there that you may have agreed to, they disagree that such a rule is within the spirit of the law, and may reckon that it should not be a part of it even if it presently is. Like they explicitly mention how they're reflecting on the idea behind it all, rather than producing a legal analysis. Or at least that's how I read it. I'm sure GP will clarify shortly.
- fc417fc802 1y agoMore precisely, the reasoning hinges on the assertion that "Your models are derived works." which I doubt is so cut and dry. There are many processes that take external information as input. That alone clearly can't be sufficient to established that something is unambiguously a derived work.
- SubiculumCode 1y agoI read a copyrighted book, it is lossy encoded into the weights of my brain. Am I a derivative work now? No. If that book inspires me to write another book in its genre, it will also not be a derivative work unless it adheres too closely to the original book.
- CaptainFever 1y agoNitpick: Can a license restrict you? I thought it only gave you additional rights, should you choose to accept it, but can't take away rights you have (e.g. the ability to make parodies from it). The restriction comes from IP laws themselves.
- MengerSponge 1y ago[flagged]
- nick__m 1y agoPowerful tools that would not exist otherwise !
- michaelt 1y agoIt might seem hypocritical, but you can take the tool and still support throwing the creator in jail. Rockets? Pretty cool. Wernher von Braun? Not cool.
- fc417fc802 1y agoIn this case IP law is clearly extremely broken. Rather than "carrying OpenAI's water" how about "advocating for fixing broken things"? The alternative is carrying Mickey Mouse's water and I've certainly no motivation to do that.
- bandrami 1y agoPublic performance of a written work has different rules than reproducing text or images. But me drawing Superman would absolutely be violating DC's copyright. They probably wouldn't care since my drawing would suck, but that's not the legal issue.
- perching_aix 1y agoYou remind me to all the shitty times in literature class where I had to rote memorize countless works from a given author (poet), think 40, then take a test identifying which poem each of the given quotes was from. The little WinForms app I wrote to practice for these tests was one of the first programs I've ever written. I guess in that sense it's also a fond memory. I miss WinForms. Good thing they were public (?) works, wouldn't wanna get sued [0] for possibly being a two legged copyright infringement. Or should I say having been, since naturally I immediately erased all of these works from my mind just days after these tests, even without any legal impetus. Edit: thinking about it a bit more, you also remind me to our midterm tests from the same class. We had to produce multiple page long essays on the spot, analyzing a select work... from memory. Bonus points for being able to quote from it, of course. Needless to say, not many original thoughts were featured in those essays, not in mine, not in others' - the explicit expectation was that you'd peruse the current and older textbooks to "learn (memorize) the analysis" from, and then you'd "write about it in your own words", but still using technical terms. They were pretty much just tests in jargon use, composition, and memorization, which is definitely a choice of all time for a class on literature. But I think it draws an interesting perspective. At no point did we ever have to actually demonstrate a capability in literary analysis of our own, or was that capability ever graded, for example. But if you only read our essays, you'd think we were great at it. It was mimicry. Doesn't mean we didn't end up developing such a capability though. [0] https://youtu.be/-JlxuQ7tPgQ https://youtu.be/-JlxuQ7tPgQ
- thaumasiotes 1y ago> I miss WinForms. WinForms is still around. There have been further technologies, but as far as I can tell the current state of things is basically just a big tire fire and about the best you can do is to ignore all of them and develop in WinForms. Is there a successor now?
- perching_aix 1y agoI miss WinForms in the sense that I don't use it anymore (and have no reason to), not in the sense that it's been deprecated. It did fall out of fashion somewhat though, as far as I'm aware it's been replaced by WPF in most places.
- rockemsockem 1y agoI think I big part of copyright law is whether the thing created from copyrighted material is a competitor with the original work, in addition to whether it's transformative. LLMs are OBVIOUSLY not a replacement for the books and works that they're trained on, just like Google books isn't.
- tossandthrow 1y agoWhy not? Imagine a story teller app that is instructed in narrating a story the follows Harry Potter 1 - I would expect that there are already a ton of these apps out there.
- rockemsockem 1y agoThat's not the same as the LLM itself though. That's an LLM plus specific instructions which would likely need to include a fair number of details from the books
- crmd 1y agoAuthors are getting busted[1] on a regular basis publishing LLM-generated or augmented novels that plausibly compete commercially with human-written books. If the LLM was trained on any books in the same genre, it seems like a clear violation. 1. For example earlier this month: https://www.reddit.com/r/fantasyromance/comments/1ktrwxj/fantasyromance_author_called_out_for_using_ai/ https://www.reddit.com/r/fantasyromance/comments/1ktrwxj/fan...
- rockemsockem 1y agoBut that's not the LLM itself, it's an output of the LLM. The distinction matters I believe
- CaptainFever 1y agoI think what the GP meant was that it doesn't compete with that specific work it was trained on. That is, if you want to read Harry Potter, you'd rather buy it (or get it from Anne) than try to wrangle it out of an LLM. Therefore, it doesn't compete with the original work. IANAL, though.
- anothernewdude 1y agoOne day they'll use these cases to sue people who have read books, because they can make immediately identifiable if imperfect copies of the works.
- tossandthrow 1y agoWhat? Can we copy you brain in the billions and load it into a phone like a commodity?
- CaptainFever 1y agoWhat does that matter? (Also: you never know, that might be possible some day.) You can still infringe copyright without a computer or printing press; just write out a book that you remember and distribute it.
- cortesoft 1y agoJust because an LLM has the ability to infringe copyright doesn’t mean everything it does infringes copyright.
- fluidcruft 1y agoIf it contains the copyrighted material, copyright laws apply. Being able to produce the content demonstrates pretty conclusively that it contains the copyrighted material. The only real question is whether it's possible to prevent the system from generating the copyrighted content. A strange analogy would be some sort of magical BluRay that plays novel movies unless you enter the decryption key. And somehow you would have to prevent entering those keys.
- echelon 1y ago> If it contains the copyrighted material, copyright laws apply. Not so fast! That hasn't been tested in court or given any sort of recommendation by any of the relevant IP bodies. And to play devil's advocate here: your brain also contains an enormous amount of copyrighted content. I'm glad the lawyers aren't lobotomizing us and demanding licensing fees on our memories. I'm pretty sure if you asked me to sit under an MRI and recall scenes from movies like "Jurassic Park", my visual cortex would reconstruct scenes with some amount of fidelity to the original. I shouldn't owe that to Universal. (In a perfect world, they would owe me for imprinting and storing their memetic information in my mind. Ad companies and brands should for sure.) If I say, "One small step for man", I'm pretty confident that the lot of you would remember Armstrong's exact voice, the exact noise profile of the recording, with the precise equipment beeps. With almost exacting recall. I'm also pretty sure your brains can remember a ginormous amount of music. I know I can. And when I'm given a prediction task (eg. listening to a song I already know), I absolutely know what's coming before it hits.
- fluidcruft 1y agoA better analogy is memorized poems or music and I think there's actually considerabe case law on whether performing songs infringe on the author's copyright. Movies are maybe less intuitive because most people won't reproduce a movie. There supposedly are people with "photographic memories" for example. And infringement is possibly not so much the capacity as what is actually done. Someone with a photographic memory could duplicate a movie but unless they actually do their capacity is not infringement. But you also have to consider that if it was viewed "lawfully" in the first place then the copyright holder has decided to let people view them. So the act of a brain seeing Jurassic Park is what the copyright holder authorized. Generally I don't think copyright holders have agreed to have LLM ingest their works. Plays and opera are maybe more similar, because people to copy productions etc. But those don't feature encryption so the blob of unknown data doesn't feature in the analogy.
- jonplackett 1y agoI don’t think it’s that simple. Last I remember, whether it is ‘transformative’ is what’s important. https://en.m.wikipedia.org/wiki/Transformative_use https://en.m.wikipedia.org/wiki/Transformative_use Eg. Google got away with ‘transforming’ books with Google books. https://www.sgrlaw.com/google-books-fair-transformative-use/ https://www.sgrlaw.com/google-books-fair-transformative-use/
- karaterobot 1y agoWhere in the article do the authors say this puts anything to rest? Here is their conclusion: > Our results complicate current disputes over copyright infringement, both by rejecting easy claims made by both sides about how models work and by demonstrating that there is no single answer to the question of how much a model memorizes I wonder if this is the sort of article that people will claim supports their side, even that it ends the debate without a knockout blow to the other side, when the actual article itself isn't making any such claim. I'm sure you read the entire article before commenting, but I would strongly recommend everyone else does as well.
- bdbenton5255 1y agoSuchir Balaji did not die in vain.
- flir 1y agoSize of LLM: <64Gb. Size of training data: fuck knows, but The Pile alone is 880Gb. Public github's gonna be measured in Tb. A Common Crawl is about 250Tb. There's physically not enough space in there to store everything it was trained on. The vast majority of the text the chatbot was exposed to cannot be pulled out of it, as this paper makes clear. I'm guessing that the cases where great lumps of copyright text can be extracted verbatim are down to repetition in the training data? There's probably a simple fix for that. (I'm only talking about training here. The initial acquisition of the data clearly involved massive copyright infringement).
- jrm4 1y agoYou'd have a very hard time legally distinguishing this from "compressing a copyrighted work" though.
- vintermann 1y agoTo get out the original data from a compressed file, you just need to know the algorithm used (and for almost all formats, the file tells you). To get out the original data from an LLM, you need to supply... the original data. Or at least, a big chunk of it. The actually copyrightable chunk of it, arguably, since what a LLM can generate on its own is only its most predictable, unoriginal, generic chunks. Things it's seen a thousand times. Which may turn out to be an uncomfortably high % of most creative works.
- flir 1y agoI wouldn't, because the vast majority of the copyrighted works it was trained on are not present in the model, and the model can't be persuaded to spit them out at any reasonable level of fidelity (as the paper points out). The comparatively few that are should be fixed. If you want to argue that the act of training is in itself infringing, even if it doesn't result in a copy... well, I'd enjoy seeing you make that argument.
- jrm4 1y agoI'd be happy to take "reasonable level of fidelity" + yes, the act of training itself as infringing to a jury? I feel like it's going to look way more like "feeding into a copy machine" than "teaching a toddler" or whatever.
- morkalork 1y agoIf that argument worked, anyone could use compression algorithms as a loophole for copyright infringement.
- dylan604 1y agoThe problem is if someone uses a prompt that is clearly Potter-esque, there have been examples of it returning Potter exactly. If it had never had Potter put into it, it would not be able to do that. I think the exact examples used in the past were Indiana Jones, but the point is the same.
- hnthrowaway_989 1y agoThe copyright angle is a lose lose situation. If copyrigthtists win, the outcome is even more restricted definition of "fair use" that probably is going to kill a lot of art. I don't have an alternative argument against AI art either, but I don't think you are going to like this outcome.