12 ms·
Zuckerberg approved training Llama on LibGen [pdf]
- mkmk 2y agoA partial list of ISBNs in the dataset https://github.com/psmedia/Books3Info https://github.com/psmedia/Books3Info
- consumer451 2y agoIt is very difficult for me to believe that Meta's recent political relations moves are not related to the open cases where Meta is the defendant.
- aprilthird2021 2y agoThe antitrust one is most relevant as the new party in power would be gleeful to see it broken up but otherwise disagrees with the concept of antitrust
- frob 2y ago"For my friends, everything; for my enemies, the law."
- qwertox 2y agoI don't understand your comment. This is about a lawsuit which shows that Zuckerberg OK'd the downloading and use of LibGen data. The case exists at least since mid-2023 and was in discovery phase until 13. Dec 2024. Shortly before the deadline Meta provided this new information, because they had to.
- monsieurbanana 2y agoHe's saying that he wants to pay Trump to win these lawsuits, which is a smart move as we know justice is for sale.
- tux3 2y agoThey're saying that Meta has been kowtowing to the incoming administration in hopes of getting in their good graces. Rather famously, some elements of that administration are above the criminal code, so that's not implausible.
- bamboozled 2y agoThe pivot would potentially help his cause though , would it not ?
- lupire 2y agoPP is referring to Facebook/Meta's new policy changes like banning intelligence/sanity-based insults on the Platform, but carving out an exception specifically and explicitly for transgender people as targets, and removing tampons from men's bathrooms.
- credit_guy 2y agoI guess the parent is saying that the new administration could be more business friendly in prosecuting this type of cases. It might even drop this case altogether. But only if Meta is "friendly" to the administration too.
- hatenberg 2y agoNah it’s just that Zuck watched the Barbie movie and realized Soace Karen was getting entirely too much limelight and declared a Year of Masculinity
- aprilthird2021 2y agoMaybe I'm not aware, but as Meta's Llama is free for use, why is the lawsuit against them, vs OpenAI or any other company who have assuredly (from my experience in these companies) done the exact same thing if not worse and make a profit from it?
- consumer451 2y agoI am far from a lawyer, but while it's free to use, Meta also uses Llama in their for-profit products. That is significant, is it not?
- lxgr 2y ago"Free to use" has never been legitimate excuse in copyright litigation. While commercial motives do matter (as far as I understand), being able to find evidence and bring a case practically matters even more.
- aprilthird2021 2y agoYeah I understand that, for example, pirating copyrighted works for free is still illegal and can amount to damages for the copyright holder. Idk, just rubs me the wrong way when there are companies making money on the exact same product sourced the exact same way right now, as Meta chooses to make it free and gets sued. Seems like we should be logical enough to conclude if they get sued, every similar company should be investigated and fined (if wrongdoing was found) as well.
- lxgr 2y agoMeta makes their models free to most users (not all – there's a "large commercial use" exemption!), and I do appreciate that practically. Still, they don't publish their detailed training methodology, which must have immense value for them internally. Even if they choose to never exercise the "large user exemption" in their current license, they can decide to license Llama 4 under restrictive terms (or not release weights at all – better start using Meta products if you want to gain access!) whenever it's convenient for them to do so. All of that doesn't exactly scream "public good worthy of a copyright exemption" to me, in a world where libraries, retro computing/gaming archivists and others are still continuously harassed by copyright holders.
- lxgr 2y agoIn hindsight (considering how LLMs are trained etc.) it makes total sense, but "Big Tech vs. Big Copyright" is something I didn't have on my 2020s bingo card. I wonder who will come out on top, and whether there will be any incidental improvements for consumers, but unfortunately I can imagine an "AI training exemption" all too well.
- criley2 2y agoI don't view Big Tech as being against copyright. They simply hold a position that they will not pay for something unless forced to ("make me" - a very common position for the powerful to hold). In fact, I'd argue that Big Tech is pro copyright, because once they force the copyright holder to negotiate, the cost is irrelevant to them and they build a moat around that access. For example, Google stole Reddit content for Gemini until Reddit was forced to the table, and now Google has a seemingly exclusive agreement around Reddit data for AI purposes.
- jsheard 2y ago> I don't view Big Tech as being against copyright. They simply hold a position that they will not pay for something unless forced to Yep, the contradiction between them feeling entitled to use anything they want for training, while simultaneously having license terms which forbid using the output of their models to train other models is pretty glaring. Information wants to flow freely but only in one direction apparently.
- lxgr 2y ago> having licenses which forbid using the output of their models to train other models I haven't been following it closely, but aren't there already court rulings saying that generative AI output by itself is not copyrightable?
- jsheard 2y agoThere's not much caselaw as to whether those terms are actually enforceable yet, but it at least indicates what they want to happen.
- bagels 2y agoLibGen is the most generic name ever, had to look it up. Turns out that LibGen is a collection of pirated books. https://en.m.wikipedia.org/wiki/Library_Genesis https://en.m.wikipedia.org/wiki/Library_Genesis
- A_D_E_P_T 2y agoIt's not just a collection, it's the collection. It contains almost every scientific book ever printed, for one thing. Frankly, it's a massive boon to researchers. It's like a top-tier research university library at your fingertips, and usually more convenient than the real thing.
- reddalo 2y agoAlso free. That helps. But the sad state of the affairs is that if Aaron Swartz does it, he ends up dead; if Meta does it, everything is fine.
- A_D_E_P_T 2y agoA lot of people would gladly pay. I'm a paying subscriber to Anna's Archive, which vastly improves the experience of that site. (It's borderline unusable without a subscription.) Thing is, the Elsevier/Springer model makes it incredibly difficult to pay them. With single papers or book chapters in the $30-40 range, an afternoon's research can easily cost $600. (Note that the authors and reviewers don't get royalties on this, and the Editor-in-Chief of any given journal usually only makes a small stipend!) There are services like DeepDyve, but they're intentionally gimped and difficult to use, because their user interface is 100% built around preventing you from downloading or screenshotting the papers you "rent"! If the publishers set up a $100/month all-open-access program, and if the experience were at least halfway decent, I'd bet that a lot of people sign up. And that's not cheap!
- mistercheph 2y agoFunny that the world where almost all human knowledge and art is free and accessible for everyone exists in parallel to one where articles about which McDonalds meal are you are paywalled, and funny which world civilized nations have chosen in order to protect The Suite Life of Zack & Cody and all the artists whose livelihoods depend on reruns of iCarly.
- agilob 2y agoMy comment from 2 months ago saying: >FBI should go after OpenAI and Sam Altman the same way they went after Aaron Swartz. Should now be expanded to: FBI should go after OpenAI, Sam Altman and Meta the same way they went after Aaron Swartz.
- portaouflop 2y agoNo they should go million times harder after them given that they built billion dollar businesses on top of this - but maybe that’s tue exact reason why they don’t?
- ALittleLight 2y agoDisagree. They were wrong to go after Aaron Swartz and would be wrong to go after OpenAI or Meta. The federal government should instead require any university receiving federal supports, support, or students on federal loans to require anyone associated with the university to publish only in completely open journals - providing unrestricted access to all. Then the government should void whatever intellectual property protections currently fetter academic literature.
- portaouflop 2y agoFair point and I agree with you on that - I guess my point was that Aaron’s “crime” pales in comparison to what they do. But yea copyright is broken and is holding back humanity as a whole
- Havoc 2y agoThat was my first thought too, though wiki suggests that at least a portion of Aaron's sentence wasn't about piracy but unauthorized access. i.e. "hacking" So not a direct analog
- bagels 2y agoHe was sentenced? I thought he didn't make it to trial?
- themerone 2y agoArtists having been imitating the works of others for thousands of years. Why is it so controversial to have an AI do the same?
- threeducks 2y ago[dead]
- pieix 2y agoThe controversy here is that LibGen doesn't legally distribute its content. Mass-scale training on pirated content is... legally murky, to say the least.
- energy123 2y agoI do think a new way of thinking about copyright is needed for AI. Allow tech firms to train on all material, but there should be an AI tax that serves as compensation back to the public commons for what was taken from it and privatized. The status quo favors large players who can navigate the legal system.
- CuriouslyC 2y agoTraining on copyright content as research is entirely fair use. Just require that fair use defense to hinge on that research being public, i.e. that it's only a defense of open weight models. A tax on AI is stupid because the big players can dodge taxes well and have the ear of power now, so any regulation would favor them. It would only serve to prevent challengers to their dominance.
- xena 2y agoFood costs money. When the price of your labor becomes zero, you can't afford to eat.
- deleted 2y ago[deleted]
- 52-6F-62 2y ago
- Havoc 2y agoI guess the zuck would download a car... Will be interesting to see where this lands, because all outcomes seem to have significant secondary effects.
- deleted 2y ago[deleted]
- mrtksn 2y agoWe are approaching the "UBI or Guillotine" fork simply because rules and regulations work selectively. Just like with the "If we pay for copyright or business becomes impossible" defense, this is yet another wast unfairness against those who had to transfer their resources to learn a skill. Awful lot of people had hard life or got into debt for things that big tech is immune from. Or maybe we will come into the conclusion that all this works only if there's no such thing as IP, reset the playing field for everyone and if anyone wants to make money will have to actually work for it every single time. IIRC that's what's happening in China and its how they surpassed US in innovation. Technically, that's a deregulation - just not the kind of deregulation the big tech is pushing for. Maybe the next time there's a graph showing how regulations made EU lag behind, add the graph of China too to spice things up. With so many technical people out of work and promises of make the employed ones obsolete too, it can be a good idea to let people build thing instead of unfairly concentrating even more power onto kleptocratic entities.
- casey2 2y agoHow can you get the definition of fairness so backwards? Giant corporations provide literally everything you take for granted and they should be punished because you are envious? I don't get it. There is a reason everyone with over 130 IQ wants to work for them rather than starting their own companies.
- saagarjha 2y agoPeople who are smart typically have better things to do than talk about their IQs. Or sell ads, for that matter.
- Lucasoato 2y agoThey shouldn’t be punished because people are envious, they should be punished because they’re not respecting other people's intellectual property without an agreement in place. We can’t protect IPs only when that benefits big corps. We should protect them always or accept that the world is better if we go in another direction, changing the rules for everybody.
- Funes- 2y agoYes. Every "AI" company is training their software on everything, regardless of what they claim, and making millions, billions of dollars on it.
- emahhh 2y agoExactly. Not surprised at all.
- visarga 2y agoDo you mean "making cents per million tokens"? And the benefit obviously belongs to the person who prompts, because they solve a task or get help. The value of that help can be from trivial to life changing.
- freefaler 2y agoNot yet... they're not profitable still, but will be in the future (those who survive). Nvidia is making all the money from the eager investors who subsidize the "free" chatGPT & related tools.
- consumer451 2y agoYouTube was mostly a library of pirated content when Google bought it for $1.6B. Spotify began by uploading an employee's pirated MP3s, and is now valued at $92B. There are plenty of other examples. One of the ways to success is to ignore silly legal matters, build a product people want, and worry about the legality later. It's not just AI companies, the pattern is well established.
- disqard 2y agoEric Schmidt said so, and it caused a giant uproar. To me, it's just "more of the same", but apparently he said the quiet part out loud, which was somehow verboten. (Edit to add: I'm not saying "I think this is okay", but rather "this is Standard Operating Procedure for startups" -- even Reddit was seeded with fake accounts and content, to give the appearance of an active online community. This sort of hustle is a core part of SV culture, and I don't think this is going to change in a hurry.) Excerpt: ...in the example that I gave of the TikTok competitor, and by the way, I was not arguing that you should illegally steal everybody's music. What you would do if you're a Silicon Valley entrepreneur, which hopefully all of you will be, is if it took off, then you'd hire a whole bunch of lawyers to go clean the mess up, right? But if nobody uses your product, it doesn't matter that you stole all the content.
- buyucu 2y agoSince I have loads of pdfs from libgen on my laptop, I guess I should not complain about the Zuck for just this once. Also libgen is amazing. You're missing out if you're not using it.
- JTyQZSnP3cQGa8B 2y agoI'm actually training a new generation biological LLM stored in my skull, and I use audio and video material from TPB in order to achieve the best AGI ever. You can't sue me because OpenAI and Facebook are doing it, also I don't have enough money in my bank account and that's why you can't sue me. It's funny that most AI bros I have met were severely against piracy a few years ago.
- bananapub 2y agoglorious to come a day after the anniversary of Aaron Schwartz's death, chased to it by the US Feds for the a similar thing, just with with much more noble intentions than Zuckerberg's desire to rule the future.
- boramalper 2y agoAlluded multiple times in the comments already but worth being explicit: Aaron Swartz killed himself 12 years ago yesterday for facing "a cumulative maximum penalty of $1 million in fines, 35 years in prison" [0] after downloading academic journal articles, which would be only a small percentage of what's available on LibGen. Free for me, not for thee. [0] https://en.wikipedia.org/wiki/Aaron_Swartz https://en.wikipedia.org/wiki/Aaron_Swartz
- JumpCrisscross 2y ago> Free for me, not for thee Swartz was charged with 35 to 50 years, realistically faced up to 10, and was offered 6 months if he plead guilty [1]. That offer moreover wasn’t the final offer. Put another way, it’s not clear that the law is being applied to Zuckerberg differently than it was to Swartz given the law wasn’t actually ever applied to Swartz. (Or that they wouldn’t gladly trade this lawbreaking for $1mm in fines and a negotiation over penalties where the prosecution opens with 6 months jail.) The prosecutor acted inappropriately in that case; MIT, more wildly so. That doesn’t, however, carry over to a transgression of the law given we never got to that stage. [1] https://www.forbes.com/sites/forbesdev/2023/02/28/increase-website-performance-with-three-metrics/ https://www.forbes.com/sites/forbesdev/2023/02/28/increase-w...?
- inetknght 2y ago> it’s not clear that the law is being applied to Zuckerberg differently than it was to Swartz given the law wasn’t actually ever applied to Swartz Has Zuckerberg actually been charged with something with equivalent potential consequences? If not, then your statement is false on its face.
- JumpCrisscross 2y ago> Has Zuckerberg actually been charged with something with equivalent potential consequences? I didn’t say Zuckerberg has been subjected to what Swartz was. Swartz never wielded the nation-state level power of a billionaire—it’s difficult to imagine how he could be subjected to similar psychological stress. I said the law isn’t being applied to Zuckerberg (or anyone who has downloaded LibGen, for that matter) differently because the law was never applied to Swartz. Given the unpopular Swartz prosecution ended Ortiz’s career, and the lack of recent criminal copyright cases, it’s unlikely anyone would attempt to apply it as they did then. To anyone, including Zuckerberg. TL; DR If you dislike what Zuckerberg is doing, you’re probably advocating for a clarification of the law. If you like it, erm, nothing much to do here.
- resiros 2y agoI would argue that it's right call: 1) it's in the world's best interest. I am running llama locally on laptop, and the ability to have the distilled world's knowledge at your fingertips will generate much much more value than what it takes. 2) it does not 'take' any value from the book creators. No one's going to 'not buy a book' because an LLM has been trained with its content (in contrast you might argue that you are likely to not buy a book because you downloaded it from libgen). Copyright laws are not millennia-old ethical laws that everyone agrees on (like don't steal), they are a modern human construct that were created for the greater good (incentivize creation), and we should revisit them with new tech.
- lnkl 2y ago"1) it's in the world's best interest." How is pleasing Meta's shareholders in world's best interest.
- TiredOfLife 2y agoHow is using llama for free locally pleasing shareholders?
- jbentley1 2y agoThings can both please share holders and create value for users of that thing.
- ulbu 2y agowat? facebook is going to 'not buy a book' for each book it's gone through. world's best interest that one of the wealthiest companies in the world don't pay their dues? world's best interest? when we know nothing about the societal and political effects llms will have in the hands of such people? what are you rationalising about?
- edoceo 2y ago> incentivize creation Humans do that naturally (see: children) The copyright laws are to protect profit.
- jpc0 2y agoI'm at a moral impass on yhis specifically. Llama is probably one of the few LLMs that probably doesn't generate an income for Meta but I can't exactly see how other than by assisting their current ad generation. Them being open weight isn't as good a what a "proper" open source LLM would be, but vs OpenAI which likely did the same thing it's significantly better. On the other hand if copyright is enforced it should be enforced across the board, if I did the same thing while training an AI would I get the same treatment... Equal before the law and all that... On the third point, I cannot legally obtain scientific paper without very significant cost to myself. My local libraries don't have a reasonable selection and even the university libraries that will let me as a member of public or even alumni still hold membership, specifically exclude scientific papers in that membership and you need to pay per paper.
- 1vuio0pswjnm7 2y agoPDF: https://ia902305.us.archive.org/34/items/gov.uscourts.cand.415175/gov.uscourts.cand.415175.373.0.pdf https://ia902305.us.archive.org/34/items/gov.uscourts.cand.4... Text: https://www.courtlistener.com/docket/67569326/373/kadrey-v-meta-platforms-inc/ https://www.courtlistener.com/docket/67569326/373/kadrey-v-m... "Meta's request is preposterous. With one possible exception, there is not a single thing in those briefs that should be sealed." "It is clear that Meta's sealing request is not designed to protect against the disclosure of sensitive business information that competitors could use to their advantage. Rather, it is designed to avoid negative publicity." "If Meta again submits an unreasonably broad sealing request, all materials will simply be unsealed." "One final comment. Between this sealing request and assertions in Meta's opposition brief such as "[t]hat document expressly discusses torrents and seeding", Opp. at 7, the Court is becoming concerned that Meta and its counsel are starting to travel down a familiar road. See In re Facebook, Inc. Consumer Privacy User Profile Litigation, 655 F. Supp. 3d 899 (N.D. Cal. 2023)."
- elashri 2y agoThere are three positions around the usage of of shadow libraries. 1- Should we develop this argument into more discussion as society and humans around the knowledge publication and the publication industry greed and the rent-seeking business model. 2- Big Corporation shouldn't just ignore the copyright law while maintaining the strongest copyright protections and going after small folks. 3- The usual argument about how LLMs training is different from people actually using pirated textbook because it is expensive (college and learning is hard and expensive specially in places like Africa). These are different angles and I think we can try to address all of them as they are not exclusive. There are good arguments around point 3 on two sides. I don't think there is a good argument why we should allow the status quo regarding the first point though. For two, it is more complicated to even discuss specially on HN.
- miohtama 2y agoWe can rewrite copyright laws.
- oidar 2y agoA few questions I have on this: Is it possible for an LLM of llama3/sonnet3.5/GPT4o quality to be trained on freely available works? Are there other types of LLMs that can be trained on smaller data sets with comparable quality? If that is not possible, and the courts shut down training on copyrighted works - what position will the "rule following" nations be in compared to nations that don't follow those rules?
- Workaccount2 2y agoIt's not even clear that training on copyrighted data even is a breach of IP law. People start their arguments on that assumption so they have an argument, but in reality that question isn't even resolved yet, and frankly it looks like the courts will likely determine that it's not a breach of IP law to train on copyrighted data (but is a breach to output it).
- jorams 2y agoNote that training is not even relevant here. Downloading copyrighted content you don't have the right to download is illegal. Distributing content you don't have the right to distribute is illegal. Meta did both. They did so knowingly, very deliberately even. It is unambiguously copyright infringement, on a massive scale.
- jazzyjackson 2y agoHow did they get the copyrighted data? O, right, they downloaded it without permission.
- hdjjhhvvhga 2y agoWhat a coincidence - it recently started giving 502 errors. I guess only the biggest companies are entitled to training their models on the heritage of humanity for free, smaller ones and individuals don't have this right.
- gorbachev 2y agoThis is wilfull copyright infringement. Statutory damages are up to a maximum of $150,000 per work infringed. Meta's market capitalization would cover 10,333,333 works, if maximum damages were levied against it. Not that the billionaire class will ever be held to the same legal standard as the rest of us.
- ChrisArchitect 2y ago[dupe] https://news.ycombinator.com/item?id=42651007 https://news.ycombinator.com/item?id=42651007
- kristianp 2y agoAre Meta still training new llama models using libgen while this court case is happening? You'd think there would be an injunction agaimst that for the duration of proceedings.
- aprilthird2021 2y agoNo they aren't
- puppycodes 2y agoIP laws falsely equate work with value, once again stuck in the gilded cage of coorelation and causation.
- paolgiacomelli 2y agoLet's then stop calling it "Artificial Intelligence" and call it what it is, making "plagiarism software" because "It doesn't create anything, but copies existing works from existing artists and modifies them in such a way that they can escape copyright. Noam Chomsky, New York Times - March 8, 2023
- vladkens 2y agoContent distribution services are so "convenient" that it's easier to get everything in one place on a pirate torrent, LOL.