18 ms·
To be very clear on this point - this is not related to model training. It’s important in the fair use assessment to understand that the training itself is fai
by aeon_ai 1y ago
To be very clear on this point - this is not related to model training.
It’s important in the fair use assessment to understand that the training itself is fair use, but the pirating of the books is the issue at hand here, and is what Anthropic “whoopsied” into in acquiring the training data.
Buying used copies of books, scanning them, and training on it is fine.
Rainbows End was prescient in many ways.
- therobots927 1y agoIt is related to scalable mode training, however. Chopping the spine off books and putting the pages in an automated scanner is not scalable. And don't forget about the cost of 1) finding 2) purchasing 3) processing and 4) recycling that volume of books.
- Onavo 1y ago> Chopping the spine off books and putting the pages in an automated scanner is not scalable. That's how Google Books, the Internet Archive, and Amazon (their book preview feature) operated before ebooks were common. It's not scalable-in-a-garage but perfectly scalable for a commercial operation.
- knome 1y agoI remember them having a 3D page unwarping tech they built as well so they could photograph rare and antique books without hacking them apart.
- therobots927 1y agoOh I didn't know that. That's wild
- hamdingers 1y agoWe hem and haw about metaphorical "book burning" so much we forget that books themselves are not actually precious. The books that are destroyed in scanning are a small minority compared to the millions discarded by libraries every year for simply being too old or unpopular.
- johnnyanmac 1y ago>we forget that books themselves are not actually precious. Book burnings are symbolic (Unless you're in the world of Fareinheit 451). The real power comes from the political threat, not the fact that paper with words on them is now unreadable.
- wizzwizz4 1y agoWell, the famous 1933-05-10 book burning did destroy the only copies of a lot of LGBT medical research, and destroying the last copy of various works was a stated intent of Nazi book burnings.
- heavyset_go 1y agoThe real power comes from the purging of knowledge from institutions that can keep that knowledge alive. Facts, ideas and histories can all be incinerated.
- dekhn 1y agoI don't think Google Books scanner chopped off the spine. https://linearbookscanner.org/ https://linearbookscanner.org/ is the open design they released.
- mkl 1y agoNo, that's not how Google Books did it. https://en.wikipedia.org/wiki/Google_Books#Scanning_of_books https://en.wikipedia.org/wiki/Google_Books#Scanning_of_books
- debugnik 1y agoI guess companies will pay for the cheapest copies for liability and then use the pirated dumps. Or just pretend that someone lent the books to them.
- Onavo 1y agoWdym Rainbows End was prescient?
- ceejayoz 1y agoThere's a scene early on where libraries are being destructively shredded, with the shreds scanned and reconstructed as digital versions.
- nicce 1y agoI guess they must delete all models since they acquired the source illegally and benefitted from it, right? Otherwise it just encourages others to keep going and pay the fines later.
- greensoap 1y agoIn a prior ruling, the court stated that Anthropic didn't train on the books subject to this settlement. The record is that Anthropic scanned physical books and used those for training. The pirated books were being held in a general purpose library and were not, according to the record, used in training.
- nicce 1y agoThat is something which is extremely difficult to prove from either side. It is 500,000 books in total so did they really scan all those books instead of using the pirated versions? Even when they did not have much money in the early phases of the model race?
- greensoap 1y agoThe 500,000 number is the number of books that are part of the settlement. If they downloaded all of Libgen and the other sources it was more like >7Million. But it is a lot of work to determine which books can legitimately be part of the lawsuit. For example, if any of the books in the download weren't copyright (think self published) or not protected under US copyright law (maybe a book only published in Venezula) or it isn't clear who own the copyright then that copyright owner cannot be part of the class. So it seems like the 500,000 number is basically the smaller number of books for which the lawyers for the plaintiff felt they could most easily prove standing.
- reassess_blind 1y agoSo how did they profit off the pirated books?
- 1y ago
- varenc 1y ago> Rainbows End was prescient in many ways. Agreed. Great book for those looking for a read: https://www.goodreads.com/book/show/102439.Rainbows_End https://www.goodreads.com/book/show/102439.Rainbows_End The author, Vernor Vinge, is also responsible for popularizing the term 'singularity'.
- AlwaysRock 1y agoRIP to the legend. He has a lot of really fun ideas spread across his books.
- beambot 1y agoI didn't realize Vernor Vinge had passed away... Sad TIL
- gnabgib 1y agoThere was a nice discussion & nostalgia at the time (1151 points, 2024, 320 comments) https://news.ycombinator.com/item?id=39775304 https://news.ycombinator.com/item?id=39775304
- swayvil 1y agoCookie monster is his strongest work. It has a VIBE. Reminds me of permutation city
- dekhn 1y agoI got to meet him and person and tell him that his books (along with The Coming Technological Singularity) had a huge influence on my decision to go into ML. He seemed pleased. I just wish he had wrapped up the Fire Upon the Deep series.
- varenc 1y agoI got to meet him once too! Unexpectedly met him at a Media Lab demo day. I was trying to play it cool though and didn't gush to him around how he's one of my favorite authors. I regret not doing so now.
- jimmydoe 1y agoGoogle scanned many books quite a while ago, probably way more than LibGen. Are they good to use them for training?
- mips_avatar 1y agoI imagine the problem there is they primarily scanned library books so I doubt they have the same copyright protections here as if they bought them
- johanyc 1y agoIf they legally purchased them I dont think why not. IIRC they did borrow from libraries so probably not every book in Google Books
- greensoap 1y agoAnthropic legally purchased the books it used to train its model according to the judge. And the judge said that was fine. Anthropic also downloaded books from a pirate site and the judge said that was bad -- even though the judge also said they didn't use those books for training....
- xnx 1y agoAll those books were loaned by a library or purchased.
- ortusdux 1y agoThey litigated this a while ago and my understanding was that they were able to claim fair use, but I'm no expert. What I'm wondering is if they, or others, have trained models on pirated content that has flowed through their networks?
- jazzyjackson 1y agoBooks.Google.Com was deemed fair use because it only shows previews, not full downloads. Internet Archive is still under litigation iirc besides having owned a physical copy of every book they ever scanned (and keeping a copy in their warehouses) they let people read the whole thing. I’m surprised Google hasn’t hit its competitors harder with the fact that they actually got permission to scan books from its partner libraries and Facebook and OpenAI just torrented books2/books3, but I guess they have aligned incentive to benefit from a legal framework that doesn’t look to closely at how you went about collecting source material
- ants_everywhere 1y agoI wonder what Aaron Swartz would think if he lived to see the era of libgen.
- klntsky 1y agoHe died (2013) after libgen was created (2008).
- arcanemachiner 1y agoYeah but did he die before anybody actually knew about it?
- jay_kyburz 1y agoIs lib still around anymore. I can't find any functioning urls
- kibae 1y agoThere are mirrors on its' wikipedia page: https://en.wikipedia.org/wiki/Library_Genesis https://en.wikipedia.org/wiki/Library_Genesis
- awesome_dude 1y agoI believe that there's a reddit sub that keeps people up to date with what URLs are, or are not, functioning at any given point in time
- pxx 1y agolibgen.help is frequently updated
- griffzhowl 1y agoRecent MEGATHREAD on status of libgen and alternatives https://www.reddit.com/r/libgen/comments/1n4vjud/megathread_status_of_libgen/ https://www.reddit.com/r/libgen/comments/1n4vjud/megathread_...
- wmf 1y agoPaying $3,000 for pirating a ~$30 book seems disproportionate.
- robterrell 1y agoWith the per-item limit for "willful infringement" being $150,000, it's a bargain.
- gpm 1y agoAnd a low end of $750/item.
- soks86 1y agoNot if 100 companies did it and they all got away. This is to teach a lesson because you cannot prosecute all thieves. Yale Law Journal actually writes about this, the goal is to deter crime because in most cases damages cannot be recovered or the criminal will never be caught in the first place.
- vlovich123 1y agoIf in most cases damages cannot be recovered or the criminal will never be caught in the first place, then what is the lesson being taught? Doesn't that just create a moral hazard where you "randomly" choose who to penalize?
- jdkee 1y agoIt's about sending a message.
- vlovich123 1y agoThe message being you’ll likely get away with it?
- npunt 1y ago
- zer00eyz 1y ago> It’s important in the fair use assessment to understand that the training itself is fair use, I think that this is a distinction many people miss. If you take all the works of Shakespeare, and reduce it to tokens and vectors is it Shakespeare or is it factual information about Shakespeare? It is the latter, and as much as organizations like the MLB might want to be able to copyright a fact you simply cannot do that. Take this one step further. IF you buy the work, and vectorize it, thats fine. But if you feed it in the vectors for Harry Potter so many times that it can reproduce half of the book, it becomes a problem when it spits out that copy. And what about all the other stuff that LLM's spit out? Who owns that. Well at present, no one. If you train a monkey or an elephant to paint, you cant copyright that work because they aren't human, and neither is an LLM. If you use an LLM to generate your code at work, can you leave with that code when you quit? Does GPL3 or something like the Elastic Search license even apply if there is no copyright? I suspect we're going to be talking about court cases a lot for the next few years.
- arcticfox 1y ago> And what about all the other stuff that LLM's spit out? Who owns that. Well at present, no one. If you train a monkey or an elephant to paint, you cant copyright that work because they aren't human, and neither is an LLM. This seems too cute by half, courts are generally far more common sense than that in applying the law. This is like saying using `rails generate model:example` results in a bunch of code that isn't yours, because the tool generated it according to your specifications.
- kennywinker 1y agoThe example is a real legal case afaik, or perhaps paraphrased from one (don’t think it was a monkey - an ape? An elephant?). I’d guess the legal scenario for `rails generate` is that you have a license to the template code (by way of how the tool is licensed) and the template code was written by a human so licensable by them and then minimally modified by the tool.
- gpm 1y ago
- mdp2021 1y ago> Buying used copies of books It remains deranged. Everyone has more than a right to freely have read everything is stored in a library. (Edit: in fact initially I wrote 'is supposed to' in place of 'has more than a right to' - meaning that "knowledge is there, we made it available: you are supposed to access it, with the fullest encouragement").
- mvdtnz 1y agoHuh?
- riquito 1y agoI think he implies that because one can borrow hypothetically any book for free from a library, one could use them for legal training purposes, so the requirement of having your own copy should be moot
- jazzyjackson 1y agoLibraries aren’t just anarchist free for alls they are operating under licensing terms. Google had a big squabble with the university of Illinois Urbana Champaign research library before finally getting permission to scan the books there. Guess what, Google has the full text but books.google.com only shows previews, why is an exercise to the reader literally
- gpm 1y agoLibraries are neither anarchist free for alls nor are they operating under licensing terms with regards to physical books. They're merely doing what anyone is allowed to with the books that they own, loaning them out, because copyright law doesn't prohibit that, so no license is needed.
- lotsoweiners 1y agoYup. And if Anthropic CEO or whoever wants to drive down to the library and check out 30 books (or whatever the limit is), scan them, and then return them that is their prerogative I guess.
- shortformblog 1y agoThanks for the reminder that what the Internet Archive did in its case would have been legal if it was in service of an LLM.
- kennywinker 1y agoLLM’s are turning out to be a real get-out-of-legal-responsibilities card, hey?
- yoyohello13 1y agoMany things become legal when the perpetrator has money.
- mountainriver 1y agoThe golden rule: He who has the gold makes the rules
- deleted 1y ago[deleted]
- 1gn15 1y agoThis is a good soundbite but doesn't make sense. The Internet Archive had to pay for redistributing copyrighted materials. Anthropic just paid too. (Note: redistributing != training)
- tpmoney 1y agoI like the IA as much as anyone else, but surely there's a significant difference between distributing literal word for word exact copies of copyrighted material and distributing statistical indexes about copyrighted material right?
- florbnit 1y ago> Buying used copies of books, scanning them, and training on it is fine. Buying used copies of books, scanning them, and printing them and selling them: not fair use Buying used copies of books, scanning them, and making merchandise and selling it: not fair use The idea that training models is considered fair use just because you bought the work is naive. Fair use is not a law to leave open usage as long as it doesn’t fit a given description. It’s a law that specifically allows certain usages like criticism, comment, news reporting, teaching, scholarship, or research. Training AI models for purposes other than purely academic fits into none of these.
- bigmadshoe 1y agoBuying used copies of books, scanning them, training an employee with the scans: fair use. Unless legislation changes, model training is pretty much analogous to that. Now of course if the employee in question - or the LLM - regurgitates a copyrighted piece verbatim, that is a violation and would be treated accordingly in either case.
- bink 1y ago> Buying used copies of books, scanning them, training an employee with the scans: fair use. Does this still hold true if multiple employees are "trained" from scanned copies at the same time?
- bigmadshoe 1y agoSimultaneously I guess that would violate copyright, which is an interesting point. Maybe there's a case to be made there with model training. Regardless, the issue could be resolved by buying as many copies as you have concurrent model training instances. It isn't really an issue with training on copyrighted work, just a matter of how you do so.
- arduanika 1y agoComputers aren't people. And analogies aren't laws.
- rchaud 1y ago> Buying used copies of books, scanning them, and training on it is fine. But nobody was ever going to that, not when there are billions in VC dollars at stake for whoever moves fastest. Everybody will simply risk the fine, which tends to not be anywhere close to enough to have a deterrent effect in the future. That is like saying Uber would have not had any problems if they just entered into a licensing contract with taxi medallion holders. It was faster to just put unlicensed taxis on the streets and use investor money to pay fines and lobby for favorable legislation. In the same way, it was faster for Anthropic to load up their models with un-DRM'd PDFs and ePUBs from wherever instead of licensing them publisher by publisher.
- ReFruity 1y ago> But nobody was ever going to that If this is a choice between risking to pay 1.5 billion or just paying 15 mil safely, they might.
- crote 1y agoOption 1: $183B valuation, $1.5B settlement. Option 2: near-$0 valuation, $15M purchasing cost. To an investor, that just looks like a pretty good deal, I reckon. It's just the cost of doing business - which in my opionion is exactly what is wrong with practices like these.
- fn-mote 1y ago> which in my opionion is exactly what is wrong with practices like these. What's actually wrong with this? They paid $1.5B for a bunch of pirated books. Seems like a fair price to me, but what do I know. The settlement should reflect society's belief of the cost or deterrent, I'm not sure which (maybe both). This might be controversial, but I think a free society needs to let people break the rules if they are willing to pay the cost. Imagine if you couldn't speed in a car. Imagine if you couldn't choose to be jailed for nonviolent protest. This isn't some case where they destroyed a billion dollars worth of pristine wilderness and got off with a slap on the wrist.
- dtagames 1y agoThis is excellent news because it means that folks who pay for printed books and scan them also can train with their content. It's been said already that we've already trained on "the entire (public) internet." Printed books still hold a wealth of knowledge that could be useful in training models. And cheap, otherwise unwanted copies make great fodder for "destructive" scanning where you cut the spine off and feed it to a page scanner. There are online services that offer just that.
- gnabgib 1y agoTo be even more clear - this is a settlement, it does not establish precedent, nor admit wrongdoing. This does not establish that training is fair use, nor that scanning books is fine. That's somebody else's battle.
- djoldman 1y agoRight, the settlement doesn't. However, the judge already ruled on the only important piece of this legal proceeding: > Alsup ruled in June that Anthropic made fair use of the authors' work to train Claude...
- stingraycharles 1y agoWhich is very important for e.g. the NYT lawsuit against OpenAI. Basically there’s now precedent that training AI models on text and them producing output is not copyright infringement.
- jkaplowitz 1y agoJudge Alsup’s ruling is not binding precedent, no.
- dragonwriter 1y agoThe ruling also doesn’t establish precedent, because it is a trial court ruling, which is never binding precedent, and under normal circumstances can’t even be cited as persuasive precedent, and the settlement ensures there will be no appellate ruling.
- pera 1y agoOn top of that this was just one case in the US. It's honestly a bit ridiculous how some Americans seem to believe that when one random judge from their country rules something that instantly turns into an international treaty that every country on Earth must accept.
- 1y ago
- djoldman 1y agoYes, the ruling was a massive win for generative AI companies. The settlement was a smart decision by anthropic to remove a huge uncertainty. 1.5 is not small, but it won't stop them or slow them significantly.
- GabeIsko 1y agoYes, but the cat is out of the bag now. Welcome to the era of every piece of creative work coming with an EULA that you cannot train on it. It will be like clearing samples.
- Joel_Mckay 1y agoMany already did this years ago for game resources on iClone, Unity, and UE. There are also a lot of usage rules that now make many games unfeasible. We dug into the private markets seeking less Faustian terms, but found just as many legal submarines in wait... "AI" Plagiarism driven projects are just late to the party. =3
- amradio1989 1y agoI think the jury is still out on how fair use applies to AI. Fair use was not designed for what we have now. I could read a book, but its highly unlikely I could regurgitate it, much less months or years later. An LLM, however, can. While we can say "training is like reading", its also not like reading at all due to permanent perfect recall. Not only does an LLM have perfect recall, it also has the ability to distribute plagiarized ideas at a scale no human can. There's a lot of questions to be answered about where fair use starts/ends for these LLM products.
- stickfigure 1y ago> Not only does an LLM have perfect recall This has not been my experience. These days they are pretty good at googling though.
- baby_souffle 1y agoThey do not have perfect recall unless you provide them a passage in the current context and then ask them to quote it. The 'lossy encyclopedia' analogy is quite apt
- heavyset_go 1y ago> I could read a book, but its highly unlikely I could regurgitate it, much less months or years later. And even if one could, it would be illegal to do. Always found this argument for AI data laundering weird.
- elpool2 1y agoBut there is a difference between “illegal to regurgitate it” and “illegal to remember it”. IIRC in this case that settled the judge had ruled on “remember” (fair use) but not on the other.
- tpmoney 1y agoHas anyone actually made the argument that having an AI regurgitate a word for word copy of an otherwise copyrighted work is fair use? Or have they made the argument that training the AI is transformative and fair use, and using that AI to generate works that are similar but not duplications of the copyrighted work is fair use? A xerox machine can reproduce an exact copy of a book if you ask it to, but that doesn't make a xerox machine inherently a copyright violation, nor does it make every use of a xerox machine a violation of copyright, even when the inputs are materials under copyright. So far the judge in this case has ruled that training an AI is sufficiently transformative, and that using legally acquired works for that purpose is not a violation of copyright. That outcome seems entirely unsurprising given the years of case law around copyright and technology that can duplicate copyrighted works. See the aforementioned xerox machines, but also CD ripping, DVRs, VHS recording of TV shows, audio cassette recording, emulators, the Java API lawsuit and also the Google Books lawsuit.
- Nursie 1y ago> pirating of the books is the issue I have an author friend who felt like this was just adding insult to injury. So not only had his work been consumed into this machine that is being used to threaten his day job as a court reporter, not only was that done without seeking his permission in any way, but they didn’t even pay for a single copy. Really embodies raising your middle finger to the little guy while you steamroll him.
- Lu2025 1y agoExactly this. It's only us peons who will be prosecuted under the current copyright laws. The rich and well connected will base their entire business on blatant theft and will get away with it.
- skywhopper 1y agoThen shouldn’t they be liable for at least 25 times this amount?
- zizee 1y agoHas it been decided that training models is fair use? Has it been decided in all jurisdictions?
- h2zizzle 1y agoThe Librareome project was about simply scanning books, not training AI with them. And it was a matter of trying to stop corporations from literally destroying the physical books in the process. I don't know that this is applicable.
- caconym_ 1y ago> It’s important in the fair use assessment to understand that the training itself is fair use IIUC this is very far from settled, at least in US law.
- doctorpangloss 1y agoYes, but if you are predisposed for some reason to think that Anthropic "won" this case, then you're going to believe all sorts of things.
- anon-3988 1y ago> Buying used copies of books, scanning them, and training on it is fine. Awesome, so I just need enough perceptrons to overfit every possible copyrighted works then?
- safety1st 1y agoNevertheless, a crime is a crime. I'm so over this shift in America's business model. Original Silicon Valley model, and generally the engine of American innovation/growth/wealth equality for 200 years: Come up with a cool technology, build it in your garage, get people to fund it and sell it because it's a better mousetrap. New model: Still come up with a cool idea, still get it funded and sold, but the idea involves committing crime at a staggering scale (Uber, Google, AirBnB, all AI companies, long list here), and then paying your way out of the consequences later. Look some of these laws may have sucked, but having billionaires organize a private entity that systematically breaks them and gets off with a slap on the wrist, is not the solution. For one thing, if innovation requires breaking the law, only the rich will be able to innovate because only they can pay their way out of the law. For another, obviously no one should be able to pay their way out of following the law! This is basic "foundations of society" stuff that the vast majority of humans agree on in terms of what feels fair and just, and what doesn't. Go to a country which has really serious corruption problems, like is really high on the corruption index, and ask the people there what they think about it. I mean I live in one and have visited many others so I can tell you, they all hate it. It not only makes them unhappy, it fills them with hopelessness about their future. They don't believe that anything can ever get better, they don't believe they can succeed by being good, they believe their own life is doomed to an unappealing fate because of when and where they were born, and they have no agency to change it. 25 years ago they all wanted to move to America, because the absence of that crushing level of corruption was what "the land of opportunity" meant. Now not so much, because America is becoming more like their country. This timeline ends poorly for all of us, even the corrupt rich who profit from it, because in the future America will be more like a Latin American banana republic where they won't be able to leave their compounds for fear of getting Luigi'ed. We normal people get poverty, they get fear and death, everyone loses. The social contract is collapsing in front of our eyes.
- rapind 1y agoWelcome to the age of grift.
- Spooky23 1y agoI agree with you, except that you’re too positive. The United States is already a banana republic. The federal courts are a joke - the supreme court now has at least one justice whose craven corruption is notorious — openly accepting material value (ie bribes) from various parties. The district courts are being stuffed with Trump appointees with the obvious problems that go with that. The congress is supine. Obviously they cannot act in any meaningful capacity. We don’t have street level corruption today. But we’ve fired half the civil service, so I doubt that will continue.
- outside415 1y ago[dead]
- mongol 1y ago> It’s important in the fair use assessment to understand that the training itself is fair use Is this completely settled legally? It is not obvious to me it would be so
- omnicognate 1y agoIt is not.
- lIl-IIIl 1y agoDo they actually need to scan the book? Or can they buy the book, and then use the pirated copy?
- whateveracct 1y agoOkay, so the blame for the offense was laundered..
- rvnx 1y agoIt should not be fine to train on them, because you are creating derivative works, exactly like when you deal with music.
- mseri 1y agoMETA did pirate basically all books in Anna’s archive but if I remember correctly they just whispered a a cried sorry and it ended up as that. Why are they also not asked to pay?
- TimByte 1y agoYou can't grab pirated stuff and then hope fair use magically sanitizes it
- nickpsecurity 1y agoI don't believe that's true. Most work I've read on fair use suggests it has to be a small amount, selectively used, substantially transformed, and not compete with content creators. These AI's training are the opposite of all that. I was surprised of a ruling like this but Alsup is a unique judge. Additionally, sharing copyrighted works without permission... the data sets or data lakes... is its own tort. You're guilty just for sharing copies before even training. Some copyrighted works are also commercial, copyright with ban on others' commercial use, and patented. Some are NDA'd but 3rd party leaked them. Sources like Common Crawl probably have plenty of such content. Additionally, there's often contractual terms of use on accessing the content. Even Singapore's and others laws allowing training on copyrighted content usually require that you lawfully accessed that content in the first place. The terms of use are the weakest link there. I'd like to see these two issues turned by law into a copyright exception that no contract can override. It needs to specifically allow sharing scraped, publicly-visible content. Anything you can just view or download which the copyright owner put up. The law might impose or allow limits on daily scraping quantity, volume, etc to avoid damage scrapers are doing.
- aprilthird2021 1y ago> the training itself is fair use Sure, training by itself isn't worth anything. Distributing and collecting payment for the usage of a trained model which may violate copyright, etc. that's still an open legal question and worth billions as well.
- spion 1y agoIts not settled whether AI training is fair use.
- Fire-Dragon-DoL 1y agoI keep thinking,if they bought ebooks,would that be fine or is this required to be paper books? If it doesn't work with ebooks, the world is going to be a nightmare
- downrightmike 1y agoThe RIAA should step in and get the money that publishers deserve. Talking millions per book and extra to make sure the pirates learned their lesson. And prison for the management.