4 ms·
I don’t understand how this can be true if set 2 contains a complete copyrighted work (say, a book) that the copyright owner hasn’t approved for such distributi
by layer8 3y ago
I don’t understand how this can be true if set 2 contains a complete copyrighted work (say, a book) that the copyright owner hasn’t approved for such distribution. Unless I misunderstand and the “process[ing] for language modeling” is an entirely irreversible process.
- doctorpangloss 3y ago[flagged]
- layer8 3y agoYour last sentence is a non-sequitur.
- doctorpangloss 3y agoAll of this conversation aside, anyone who is saying "fair use" concedes that their use more or less would be explicitly unauthorized, but because it advances "some" "goals" in this country, it should be allowed anyway. It is a necessarily adversarial stance but it belies that the author the Pile sincerely wants to advance human progress in an idiosyncratic way. I think this is what pisses off the commenters here, they want to take sides, but like, who cares. > I don’t understand how this can be true if set 2 contains a complete copyrighted work (say, a book) that the copyright owner hasn’t approved for such distribution. Everything you're saying could be true, but also meaningless. Meaningless in what sense? In my personal opinion, here are some meaningful things: - Economic meaning: Is the Pile making anyone suffer, in an intellectually honest sense? No. Is the way it is being used making anyone suffer? No. Would anyone use the Pile if they had to pay for it? No. Would some tiny amount of money paid for it matter? No, not to anyone. The Pile is but one of many things that are going on in this pareto-inefficient world that we gain little economically by fucking with. - Meaning in the sense of human progress. Is the Pile sincerely, substantively helping advance creative and scientific progress? Yes. Can the progress it promotes coexist peacefully, in this current status quo, with selling books on Amazon, which can also advance progress? Yes. Is there any evidence that this non-economic progress, such as sharing important stories or points of view meaningfully through text content, is harmed by the Pile? No. The following ways of looking at this issue are meaningless: - Psychological meaning is meaningless. Feelings that are more strongly felt by more favored parties, like beloved authors versus impetuous programmers, are not more valid. Stakeholders like authors merely exercising their collective bargaining power over something that doesn't actually make them worse off is meaningless too. Social media drama also doesn't matter. These are just opinions. I don't go out there and say I'm a lawyer, and I'm not running for Congress, and neither are you. It's crazy to me, because when you look critically at what dog you personally have in this race, like of course you want the status quo where The Pile exists, and you and the only agitators in all of this - authors' guild authors - gain nothing from nearly every dead-on-arrival framework like "Compensation, Credit and Consent" or whatever, BESIDES meaningless psychological satisfaction of winning social media arguments and flexing the favorability of authors over programmers. Imagine if we ran the whole country this way! It's radioactive. So this isn't a non-sequiter. Supporting the status quo, in this particular case, is the succinct way of saying everything here: that you can be right in ways that don't at all matter, which is okay, but you should have the insight or maybe if you are a programmer or an author or actually you are running for Congress, you have the duty to understand this stuff.
- layer8 3y agoWe are talking about the distribution of that corpus of text. Whether someone uses it for ML training or not is immaterial to the question of whether distributing it is legal.
- doctorpangloss 3y agoI am saying that you can be right that it is illegal, but nobody cares except for meaningless reasons.
- layer8 3y agoI don’t see how you can say it is meaningless if it gives anyone who downloads the pile access to countless copyrighted books. Unless you believe that copyright in general is meaningless, in which case we have nothing further to discuss.
- deleted 3y ago[deleted]
- quatrefoil 3y agoYou make a lot of categorical assertions here, but I don't think they're really all that watertight. If any individual "economically harmed in an intellectually honest sense" if PG&E overcharges every subscriber five cents on every bill? No, and a class action lawsuit against that practice doesn't make anyone better beyond "meaningless psychological satisfaction". But it's still something we do as a society, for good reasons, right? Here, you have a number of commercial entities that are poised to make a ton of money off individually small but entirely non-consensual contributions of other people. You also have a "data laundering" industry where the datasets of webpages, books, and images are published by notionally non-profit entities that are in one way or another bankrolled by the commercial players, keeping their hands clean. Framing the debate as being about "human progress" is sort of goofy in that world. We're not talking about academics tinkering with wacky ideas. We're talking about an industry that takes your work without asking and monetizes it. Yes, the sum of it is greater than the component pieces, but that doesn't mean you get to do whatever you want.
- wcrossbow 3y agoAbsolutely. This is very much like the answer you would get from a politician. It's proof by word count. Q: Why do you think X is Y? A: < 10 minute exposition that doesn't address the question > ... therefore Y.
- sva_ 3y ago> proof by word count Sorry for off-topic, but I love that expression.
- Cheer2171 3y agoBy that logic: Best to operate under the presumption that Uber's contractor-based employment model is legal, because we wouldn't be talking about any of this gig economy stuff if it weren't for that. You can argue that the law should change, but SV has this absurd idea that it somehow can't be illegal if it is a part of a successful business model.
- idiotsecant 3y ago>Best to operate under the presumption that Uber's contractor-based employment model is legal. Yes, that's how the law works. If something is in a grey area you can do it until the government tells you not to.
- wrs 3y agoA grey area by definition is where you aren’t sure whether the government has told you not to, so that’s not very useful guidance.
- deleted 3y ago[deleted]
- sangnoir 3y agoThis is the reason why Silicon Valley is losing its reality-distortion field image of disruptive innovators changing the world and being seen as the exploitative assholes on par with - or exceeding the cocaine-addled "greed is good" finance bros from the 80s. The attitude and attendant results from sloppiness are just begging for legislative clamp-downs similar to what happened with finance. You can only profitably arbitrage gaps in the law for so long before someone goes too far and there's a scandal or two, and public opinion turns against you, followed by the business-end of legislative attention.
- CamperBob2 3y agoIt's hard to call out people for being "exploitative assholes" for disrupting the legacy taxi cartels. The "exploitative assholes" at Uber appeared on the scene because the market demanded something better. As for the government, they are theoretically supposed to be responsive to the people, and the people have spoken loudly and unequivocally in favor of the "exploitative assholes."
- CharlesW 3y agoEven if the model encoding is not lossless/reversible, it's probably not true. A good place to start when thinking about fair use is the "four factors" that the U.S. legal system will consider. https://fairuse.stanford.edu/overview/fair-use/four-factors/ https://fairuse.stanford.edu/overview/fair-use/four-factors/
- layer8 3y agoSummary books for example are legal, so there is some threshold of compression where things are fine.
- kmeisthax 3y agoGoogle keeps coming up with new ways to statistically infer training set data from models. So it's not entirely lossless. At the very least, models that have been trained on a particular work are unusually good at compressing[0] those works, relative to other valid text. In terms of fair use, one of the larger factors is the 'market substitution' factor, which basically means "does this use compete with otherwise licensed uses that people would ordinarily pay for?" AI absolutely does compete with human artists for the same market. In fact, it's winning handily[1], because you don't have to pay human artists. AI art models absolutely shouldn't be trained on anything with copyright on it. The other factors don't fare much better. Nature of the original work will differ based on the plaintiff, but the purpose and character of the AI's use of that work is very much commercial. And the amount and substantiality of the use is complete and total. I don't see AI being fair use - at least, not in every one of the many, many training lawsuits currently ongoing against OpenAI and Stability. [0] Starting with any body of text, an LLM, and an empty context window, compute the next-token probabilities and take the highest one. If it matches the source text, output a 1 bit. If it doesn't, output 0 followed by the ID of the correct next token. Add the correct token to the context window and repeat until the text has been fully compressed. This produces a list of perplexities (wrong words) for the given text which can be used to guide the LLM to output the original work. [1] Hey, remember when both WotC (biggest art commissioner on the planet) and Wacom (hardware vendor that sells art tools and payment terminals[2]) both got caught using AI art after making very loud and public pledges to not do that? They both wound up buying stock photography on marketplaces that are absolutely flooded with AI trash. [2] All the credit card readers in Japan are built by Wacom, which is really funny as an artist
- IshKebab 3y agoYeah I agree. If 2 contains complete copyright works (e.g. all of Harry Potter) then "we're just using it for AI training!" stands approximately zero chance of passing the fair use test. Their assertion that it does is just wishful thinking.
- mistrial9 3y agosaid with confidence, however Silverman et al Judge explicitly rejected what you just asserted AFAIK
- whimsicalism 3y agono, she didn’t reject the claim that training on copyrighted work is infringement, merely that the outputs are not infringing simply by bearing similarity to the texts
- papercrane 3y agoYou've got it backwards. The judge in Silverman et al dismissed the claims asserting that OpenAIs output is copyright infringement. The claims for copyright infringement in the training data are still going forward, that will directly test whether it is "fair use" or not. From the ruling: > Assuming the truth of Plaintiffs’ allegations - that Defendants used Plaintiffs’ copyrighted works to train their language models for commercial profit - the Court concludes that Defendants’ conduct may constitute an unfair practice.6 Therefore, this portion of the UCL claim may proceed. https://caselaw.findlaw.com/court/us-dis-crt-n-d-cal/115818024.html https://caselaw.findlaw.com/court/us-dis-crt-n-d-cal/1158180...
- mistrial9 3y agoaha - much appreciated
- HenryBemis 3y agoI'm playing stupid now: I believe that if I ask the LLM to "display Harry Potter Book 1" and it does, word-by-word, then you're 100% right, it's copyright infringement. But, if I ask the LLM to "give me an analysis of the "Professor Severus Snape's" character and it gives me one, then I don't see the problem. So in that sense I understand the response that "they don't violate copyright" by studying the material. Again, I don't pretend to be a lawyer, and not every law has to follow my logic.
- michaelt 3y ago> Unless I misunderstand and the “process[ing] for language modeling” is an entirely irreversible process. In the case of The Pile, "processing for language modelling" means "converting epub and pdf into plain text, maybe deduplicating, maybe removing some sorts of detectably malformed files" So not a particularly lossy conversion.
- layer8 3y agoI see, thanks. Yes, in that case, I don’t see how this can possibly not constitute copyright infringement.
- bayindirh 3y agoIt's generally tucked under Fair Use doctrine because "It's for the science", until it doesn't (looking at commercial AI non-profits). Then "they're doing something amazing, they don't need permission, and the cat is already out of the bag, and similar musings". Seriously, it's both copyright infringement, and unethical. This is why I don't use any of the popular AI tools, or even AI add-ons in Evernote, Notion, etc. They all link back to the usual suspects.
- layer8 3y agoI’m talking about distributing the corpus, which by itself is not bound to any particular usage.
- bayindirh 3y agoIt's again copyright infringement. If I share a copyrighted ebook by accident, any and every cloud provider will ban my account with no warning and recourse. Open science repositories would take down the "dataset" immediately (or at least limit its access) if a copyright holder brings the matter to the eyes of the admins.
- CobrastanJorji 3y ago> "They're doing something amazing, they don't need permission, and the cat is already out of the bag" Ah, the Uber theory of law. Works surprisingly well for some reason.