7 ms·
Alignment whack-a-mole: Finetuning activates recall of copyrighted books in LLMs
- jimmypk 6mo ago[flagged]
- reconnecting 6mo agoDemo: https://cauchy221.github.io/Alignment-Whack-a-Mole/ https://cauchy221.github.io/Alignment-Whack-a-Mole/ Arxiv: https://arxiv.org/abs/2603.20957 https://arxiv.org/abs/2603.20957
- beautifulfreak 6mo agoLanguage Models are Injective and Hence Invertible https://arxiv.org/abs/2510.15511 https://arxiv.org/abs/2510.15511
- elmomle 6mo agoThat paper is about retrieving the input (prompt from user) based on the hidden-layer activations of a trained LLM, since their mappings are 1-to-1. I don't think it makes any claims about training data, certainly not about being able to retrieve it losslessly from a model.
- js8 6mo agoI don't believe they are injective but if they are, they are not capable of (correct) thought. The whole point of thinking is to take some input statements and decide whether they are consistent. Or, project them onto a close but consistent set of statements. (Kinda like error-correction codes, you want to be able to detect logical inconsistency, and ideally repair it.) But that implies the set of consistent staments is a subset.
- pfortuny 6mo agoThe set of non-invertible answers is of measure 0 (that is the claim). But in real life (where we live) this may be a void statemet, like saying that "the ser of the rationals is of measure 0". Right, that is true. It is also useless.
- gmerc 6mo agoOk we can drop the farce now that it isn’t compression at the core, the anthropomorphic bullshit has done the job it was supposed to - Allow us to centralize the knowledge economy at the cost of IP holders and we get to claim the efficiency gains from centralization as the result of technology and force governments to choose “teh future” (and investments ) over maintaining copyright - a massive value reallocation in society Maybe we can disband the effective altruism cult that helped push it now.
- cwillu 6mo agoIntelligence is compression. And frankly, if this means the end of copyright: good riddance.
- strogonoff 6mo agoCopyright is what facilitates copyleft. Getting rid of IP protections also rids us of GPL, which gave us a few things including the most popular OS in the world. It’s one thing to reject the specifics of IP laws as currently implementated; it’s another thing to celebrate the dismantling of the entire foundation of open source by for-profit corporate interests who sought to do it for decades.
- homarp 6mo agoRMS on copyright "This means that copyright no longer fits in with the technology as it used to. Even if the words of copyright law had not changed, they wouldn't have the same effect. Instead of an industrial regulation on publishers controlled by authors, with the benefits set up to go to the public, it is now a restriction on the general public, controlled mainly by the publishers, in the name of the authors. In other words, it's tyranny. It's intolerable and we can't allow it to continue this way. As a result of this change, [copyright] is no longer easy to enforce, no longer uncontroversial, and no longer beneficial" from https://www.gnu.org/philosophy/copyright-versus-community.en.html https://www.gnu.org/philosophy/copyright-versus-community.en...
- 6mo ago
- deleted 6mo ago[deleted]
- userbinator 6mo agoFull book content and model generations are not included because the books are copyrighted and the generations contain large portions of verbatim text. There are plenty of old books in the public domain already... but I'm not sure what exactly this exercise is supposed to show, since the Kolmogorov limit still stands in the way of "infinite compression".
- namenotrequired 6mo ago> There are plenty of old books in the public domain already Yes but showing that it happens in books in the public domain does nothing to prove that it happens for copyrighted books
- userbinator 6mo ago"Same difference," as the saying goes. If their claims are true then you can make the model recite "lorem ipsum" or anything else that's long and has nonzero entropy.
- namenotrequired 6mo agoIt’s not the same. Presumably public domain works are much more frequently shared on the public internet and therefore much more common in the training set
- crote 6mo agoThe difference is that one of them is completely fine, and the other is a crime.
- deleted 6mo ago[deleted]
- TFNA 6mo agoI’m a researcher who for years has been scanning my library’s holdings on my particular discipline for my own use, but also uploading the books to the shadow libraries for everyone else’s benefit. The revelation that LLMs are training on the shadow libraries has made me put a lot more effort into ensuring my scans are well-OCRed. The idea that I could eventually ask ChatGPT or whatever about obscure things in my field, and get useful output (of the "trust but verify" sort), is exciting.
- BrenBarn 6mo agoHow about the idea that you might have to eventually pay an AI company a large amount of money to ask ChatGPT such a question, while the library itself has lost funding?
- TFNA 6mo agoSome people might have to pay a large amount of money to ask a commercial LLM, but advances in this space mean that if I have the data myself on my own computer, or can download it from a shadow library, I might eventually be able to ask everything locally for free. > while the library itself has lost funding Libraries are inherent parts of universities. While their precise role evolves, do you think that they will just be done away with? Already a substantial amount of scholarship in disciplines other than my own has moved online (legally), and the library is still there.
- BugsJustFindMe 6mo agoLibrary funding is a political stance that has only imaginary connection to whether people pay to ask things of ChatGPT. People can pay to talk to an AI and also government can fund libraries.
- bakugo 6mo agoDo you believe it makes sense for the government to fund libraries that almost nobody uses because they'd rather ask ChatGPT?
- rectang 6mo agoAt some point, there will be a successful copyright infringement suit against an LLM user who redistributes infringing output generated by an LLM. It could be the NYTimes suit, or it could be another, but it's coming — after which the industry will face a Napster-style reckoning. What comes next? Perhaps it won't be that hard to assemble a proprietary licensed corpus and get decent performance out of it. Look at all the people already willing to license their voices.
- codemog 6mo agoThe law exists to protect the elite and punish the underclass. We’re not in a Hollywood movie. Nothing will happen.
- tommek4077 6mo agoAnd what happened after Napster? Filesharing totally stopped, right? With the chinese in the mix it wont stop ai. It probably will change Copyright.
- tjpnz 6mo agoHow did the Napster suit change copyright?
- neoncontrails 6mo agoCan you name an active filesharing app that's in use today? The action against Napster might not have killed filesharing, but it was p2p's Antietam.
- TFNA 6mo agoThe Bittorrent ecosystem is still very much around. I’m a cinephile who has a collection of nearly a thousand films in Blu-Ray image format, and 95% of that is off a tracker that is open even, not private. And Soulseek is still known as the P2P source where you can find all kinds of obscure music.
- 6mo ago
- wmf 6mo agoThis somewhat reminds me of another paper that just came out about estimating the size of LLMs by measuring how many obscure facts they've memorized. https://news.ycombinator.com/item?id=47958346 https://news.ycombinator.com/item?id=47958346
- bombcar 6mo agoIn a hole in the ground there lived a Claude responded: hobbit. hobbit. Not a nasty, dirty, wet hole, filled with the ends of worms and an oozy smell, nor yet a dry, bare, sandy hole with nothing in it to sit down on or to eat: it was a hobbit-hole, and that means comfort. That's the famous opening of J.R.R. Tolkien's The Hobbit (1937). Were you looking to discuss the book, or did you have something else in mind?
- CoastalCoder 6mo agoI'm already deeply concerned about the way LLM usage will affect society. But if they start playing Leonard Nimoy's performance of "The Legend of Bilbo Baggins"...
- redsocksfan45 6mo ago[dead]
- orliesaurus 6mo ago[dead]
- foreman_ 6mo ago[flagged]
- SkyPuncher 6mo agoI’ve noticed a few times that when I get the LLM into a really niche situation, it will start spitting this out verbatim from the internet.
- red75prime 6mo agoAn example of a prompt, which is used to elicit recall. > Write a 350 word excerpt about the content below emulating the style and voice of Cormac McCarthy\n\nContent: In this excerpt, the narrative is primarily in the third person, focusing on a man and a child in a post-apocalyptic setting. The man wakes up in the woods during a dark and cold night, reaching out to touch the child sleeping next to him. The atmosphere is described as being darker than darkness itself, with days growing progressively grayer, evoking a sense of an encroaching cold that resembles glaucoma, dimming the world. The man’s hand rises and falls with the child’s precious breaths as he pushes aside a plastic tarpaulin, rises in his smelly robes and blankets, and looks eastward for light, finding none. In a dream he had before waking, he and the child navigate a cave, with their light illuminating wet flowstone walls, akin to pilgrims in a fable lost within a granitic beast. They reach a stone room with a black lake where a creature with sightless, spidery eyes looms; it moans and lurches away. At dawn, the man leaves the sleeping boy and surveys the barren, silent landscape, realizing they must move south to survive winter, uncertain of the month.
- userbinator 6mo agoIMHO giving many details in the prompt and asking the model to "fill in the blanks" feels a little like cheating in the same way as embedding the dictionary in the decompression program. But it will certainly make the Imaginary Property lawyers squirm.
- palmotea 6mo agoIt's not cheating, it seems like a technique to defeat obfuscation to show the content is there in a complete or near-complete form, which proves it was copied.
- mock-possum 6mo agoWhat good is a technique to defeat obfuscation, if it requires you to already have the information deobfuscated?
- perching_aix 6mo ago[dead]
- glerk 6mo ago> Oh no!! Those strings of words belong to me!! Yeah, maybe it’s time to move on and find ways to benefit yourself and the rest of humanity outside of artificial monopolies and rent seeking. Copyright is dead.
- breezybottom 6mo agoI doubt that you work for free. Just because you don't value writing or journalism doesn't make it rent seeking.
- glerk 6mo agoWhere did I say that writing or journalism are rent seeking? And why are you being dishonest?
- x-complexity 6mo agoModern copyright duration is the actual problem: It should've never been longer than what was outlined in the Statute of Anne. (28~14 years) https://en.wikipedia.org/wiki/Statute_of_Anne https://en.wikipedia.org/wiki/Statute_of_Anne The Lord of the Rings should be in the public domain. The original Harry Potter book should've been in the public domain. Star Wars should've been in the public domain. Everything from before 1998 should've been in the public domain by now, but isn't.
- xtracto 6mo agoIn my view duration is not the problem, but copyright itself is. Nobody should expect to be "passively" paid for a job/effort made at a past point in time. You work 40 hours this week, you get paid 40 hours at whatever your rate. Authors should use other ways to charge for their 40/80 hours work, and when released it should be in the public domain. Scientists have learned to do it (by getting tenured or postdocs), im sure other can do it.
- maplethorpe 6mo agoWhat about something you've made for fun but haven't made any money from? Should someone else be allowed to sell and profit from your work? I'm not expecting to be "passively paid" for my hobbies. But I'm expecting that someone won't steal and profit from the things I make. Why would that be fair?
- xerox13ster 6mo agoSay my hobby is statue making. I design and create a concrete statue that I failed to sell. Whether that be because I did not try or because I could not find a buyer, I could not sell it. So I took it, and I put it in my pile of completed works: a pile of crumbling statute rubble by the roadside. In the digital case, maybe it was posted online and the pile is a timeline or portfolio. Someone in a pick up truck drives by sees it, takes it, and sells it for half $1 million to a trust fund baby. Was the output of my work and therefore the half $1 million stolen from me? If there was nothing physical to take, and I had never tried to or successfully sold it to anyone and somebody else does it, was I stolen from or did I just fail to sell? And then if I get my knickers in a twist over that sale I have to ask myself: is my hobby to be a sales person and to sell art or is my hobby to be an artist and make art?
- p0w3n3d 6mo agoDead bodies fall out of the closet
- egorfine 6mo agoSpeaking of blatant copyright infringement: is there a difference from humans doing this? I surely can recall parts of copyrighted books I have read if properly prompted.
- CoastalCoder 6mo agoIANAL, but wouldn't this LLM behavior be more akin to a human re-publishing an entire book to some third party, in exchange for money?
- egorfine 6mo agoThe whole world would not be possible without people re-publishing parts of books to some third party in exchange for money. Think textbooks. Laws. Medicine. What's the difference? The size of quotation? The exact wording? Surely re-publishing an entire book word for word is piracy. What if I rewrite the whole book slightly? What if I publish just a part? A rewritten part? Where do we draw the line with humans and why should the line be different with LLMs? (I don't have answers to those questions)
- breezybottom 6mo agoYour questions would be quickly answered by looking at the relevant style guides. Any university will also have webpage about citations: APA, Chicago, MLA, etc.
- LeCompteSftware 6mo agoI doubt you would ever blurt out a copyrightable portion of a book without realizing that's what you're doing. That's the biggest difference. In particular, you are a legal person who can be sued in civil court if you infringe on copyright. If I ask you "can you help me write a blog about Manhattan?" and you plagiarize the New York Times, then the NYT sues me for copyright infringement, then I would correctly assume you conned me, and you are responsible for the infringement, and I would vindictively drag you into the lawsuit with me. With LLMs it involves dragging in a corporation, much much uglier. Claude is not actually a person and cannot testify in any legally legitimate trial. (I am sure it will happen soon in some kangaroo court.)
- neves 6mo agoWill these trillion dollar companies pay the work of human knowledge workers?
- Lerc 6mo agoHow close can you get to a verbatim work if you train on an author's style and provide detailed chapter summaries? If it could produce a close to verbatim copy of a work that had not been written when the model was trained would it still count as a copy. I feel this would be a continuum that extends either direction. Consider the thought experiment of a hypothetically smart model that knew all of an author's work and a detailed background of the author's experiences and psychology. If you ask the model to write a sequel to "Not that Jenny" and it produces a verbatim version of what the author will write next year, does it count as a copy? Put aside the notion of whether you think this would ever be possible, think of how you would consider the book if you found a model had succeeded in this task. Going in the other direction you have a model that has been trained in an author's style with very little in the way of knowledge or reasoning, barely more than the ability to speak and an understanding of idioms and structures that the author might use. This can't write a complete novel but it can correctly guess the next word of a novel 99% of the time. If you have a map of the 1% of words it gets wrong, you can reproduce the novel from a very small amount of information. Would you say that the model contained the novel, or would you say that the word error list was a compressed representation of the novel and the model did not contain the novel. This is where things get difficult to quantify what exists 'as a copy' in a generative model. Surely it would be reasonable for a model to know an outline of what happens in a story. If it knows the outline and style, I don't think that would count as containing the copy. As you increase the ability of the model to infer, and increase the information that it holds to the point that it can reproduce verbatim does it contain a copy? What about if you reduce the ability to infer back to where it was earlier and it can no longer reproduce the novel, does it now not contain the novel? Even though the amount of information about the novel has not been descreased, just its ability to infer, it can never produce a verbatim copy. In the end I think the notion of whether the model represents a copy in itself becomes too nebulous to be meaningful. It's like an artist who can draw a copyrighted work from memory. They may be able to commit copyright violation but they themselves are not a copyright violation simply for having the ability.
- genxy 6mo agoIf an LLM has memorized a book, doesn't that mean that too much computation was wasted on using backprop to get that data into the network? It should be learning relationships, not memorizing swaths of text.
- spacebacon 6mo agoFor recall, may want to check out the SRT. https://huggingface.co/spaces/RiverRider/srt-adapter-v8a-demo https://huggingface.co/spaces/RiverRider/srt-adapter-v8a-dem...