6 ms·

Very cool. Guess you can just pirate 400K books stealing millions from authors as long as its a "dataset for LLMs" instead of a regular old piracy dump. No doub

Very cool. Guess you can just pirate 400K books stealing millions from authors as long as its a "dataset for LLMs" instead of a regular old piracy dump. No doubt there are books in there that took decades of experience to write and knowledge that took a lifetime to learn. Stealing is just OK as long as its for AI lol, and as we devalue writing we guarantee the quality of writing will continue to degrade.

freefaler 2y ago
Hmm... but the whole internet is also indexed by Google and they make a lot of money out of the content. Should they pay too? If this was enforced would there be search engines?
- sp332 2y ago
  It's different for websites because they are intentionally served up for free. Google had a very hard time getting a license to even index book contents. And there's an ongoing tension with paywalled websites. They do pay for some news sites, in the Google News Showcase.
- hawski 2y ago
  One of the reasons behind the backlash against Google's AMP was that they would contain the full experience. You are on a Google's OS (at least significant amount of users are), on a Google browser, using a Google search service and from there you go to Google servers to get the content. The main difference is the last step.
hawski 2y ago
The Pirate Bay should consider rebranding as The Dataset Bay. I am sure there are wild times ahead. At some time it will become outlawed in the way regular piracy is. Not because of book authors or publishers, but either because of Hollywood or AI safety being used by bigtech to limit competition.
Workaccount2 2y ago
I don't know if there is a lot of confusion, purposely misleading deception, or both, but I cannot grasp the argument for "training on copyrighted data is a violation of copyright". What I can grasp is "output of copyrighted data is violation of copyright", and the fix seems to be a straight forward dumb software contentID filter on the output.
- michaelt 2y ago
  The argument is "An artist getting inspired by a work and creating something similar is allowed, but photocopying, photographing a painting or statue, or using samples in music-making can be fraught with legal problems. One of the major dividing lines between legal and illegal is whether it's a purely mechanical process, or done by human hands. The training of LLMs and diffusion models is a purely mechanical process, more like photocopying or photographing than an artist gaining inspiration"
  - Workaccount2 2y ago
    >but photocopying, photographing a painting or statue, or using samples in music-making can be fraught with legal problems. That's part of where I think a lot of confusion is - LLM training doesn't do any of that. There is no coherent corpus of data in these models.
  - throwaway4aday 2y ago
    The training process is not purely mechanical. The amount of time that goes into selecting, cleaning, reformatting and otherwise preparing the training data is significant not to mention all of the other work involved in the actual training. If your measure is simply the amount of work done by humans then we should simply reclassify ML models as art pieces and dismiss all of the criticism as gatekeeping by people who don't understand hypermodern art.
- dialup_sounds 2y ago
  The confusion seems to be on your part, because that's not the argument being made here. The argument is that the 400k books being shared here are themselves copyrighted works that the poster probably does not have the right to be distributing, and that is not very nice of them. "It's just a dataset, bro!" being used to bulldoze people's rights to their own work might have negative consequences.
PeterStuer 2y ago
Probably copyright infringement in some jurisdictions, but it's not 'stealing'. Let's not confuse terms with specific meaning.
- zaptheimpaler 2y ago
  I don't think you understand what stealing means if you are unable to recognize that distributing free digital copies of paid books without agreed upon renumeration to the owners is stealing. It may or may not also be copyright infringement in a legal sense, but it sure is stealing in a common sense.
bilsbie 2y ago
Wait until you find out about libraries. Free books that humans are using to train the LLM’s in their heads.
- bayindirh 2y ago
  How many books a human can read, understand, tokenize, memorize in 15 minutes, 15 days, 15 months, 15 years? What's the rate of forgetting for the ingested material for human brain?
- zaptheimpaler 2y ago
  Libraries pay authors for their copies.
harshreality 2y ago
What do you think the market is for people who want to read books or articles condensed into lines of text suitable for model training? I think it's 0. Maybe a handful if pirate libraries of readable ebooks and papers didn't already exist, but even a handful of copyright violators wouldn't be a serious commercial threat to the publishers. So far, governments have been unable to do anything about pirate libraries. Complaining about AI datasets, poorly formatted for human consumption, seems misplaced. Criticizing this as "stealing" or "piracy" is a vote for a future where only big tech, or the major publishers themselves, can train models on good datasets. Nobody else will have the money or market power to license that much material.
- zaptheimpaler 2y ago
  The market for the entire contents of say OpenAI's entire codebase mangled and corrupted at parts might be 0 as well, doesn't make it not stealing. When NVIDIA's or Samsung's IP was leaked including binary firmware that is hard to read and understood by very few, that doesn't magically mean stealing and releasing it became legal.
  - harshreality 2y ago
    It doesn't magically mean anything, but you're comparing a case where there's a plausible argument for fair use (no commercial impact from the training set sharing, not done for profit, transformative), against... what specific instance are you talking about with NV or Samsung? Theft of trade secrets? Samsung leaking their own IP to chatgpt, through stupidity, isn't illegal. This doesn't even affect the major AI companies that are doing training, does it? Don't you think Google, OpenAI, Anthropic all have their own datasets by now? If they wanted to use bulk content from pirate ebook and academic paper libraries, they would've already mirrored most of those sites a long time ago.

freefaler 2y ago

Hmm... but the whole internet is also indexed by Google and they make a lot of money out of the content. Should they pay too? If this was enforced would there be search engines?

sp332 2y ago
It's different for websites because they are intentionally served up for free. Google had a very hard time getting a license to even index book contents. And there's an ongoing tension with paywalled websites. They do pay for some news sites, in the Google News Showcase.
hawski 2y ago
One of the reasons behind the backlash against Google's AMP was that they would contain the full experience. You are on a Google's OS (at least significant amount of users are), on a Google browser, using a Google search service and from there you go to Google servers to get the content. The main difference is the last step.

hawski 2y ago

The Pirate Bay should consider rebranding as The Dataset Bay. I am sure there are wild times ahead. At some time it will become outlawed in the way regular piracy is. Not because of book authors or publishers, but either because of Hollywood or AI safety being used by bigtech to limit competition.

Workaccount2 2y ago

I don't know if there is a lot of confusion, purposely misleading deception, or both, but I cannot grasp the argument for "training on copyrighted data is a violation of copyright". What I can grasp is "output of copyrighted data is violation of copyright", and the fix seems to be a straight forward dumb software contentID filter on the output.

michaelt 2y ago
The argument is "An artist getting inspired by a work and creating something similar is allowed, but photocopying, photographing a painting or statue, or using samples in music-making can be fraught with legal problems. One of the major dividing lines between legal and illegal is whether it's a purely mechanical process, or done by human hands. The training of LLMs and diffusion models is a purely mechanical process, more like photocopying or photographing than an artist gaining inspiration"
- Workaccount2 2y ago
  >but photocopying, photographing a painting or statue, or using samples in music-making can be fraught with legal problems. That's part of where I think a lot of confusion is - LLM training doesn't do any of that. There is no coherent corpus of data in these models.
- throwaway4aday 2y ago
  The training process is not purely mechanical. The amount of time that goes into selecting, cleaning, reformatting and otherwise preparing the training data is significant not to mention all of the other work involved in the actual training. If your measure is simply the amount of work done by humans then we should simply reclassify ML models as art pieces and dismiss all of the criticism as gatekeeping by people who don't understand hypermodern art.
dialup_sounds 2y ago
The confusion seems to be on your part, because that's not the argument being made here. The argument is that the 400k books being shared here are themselves copyrighted works that the poster probably does not have the right to be distributing, and that is not very nice of them. "It's just a dataset, bro!" being used to bulldoze people's rights to their own work might have negative consequences.

PeterStuer 2y ago

Probably copyright infringement in some jurisdictions, but it's not 'stealing'. Let's not confuse terms with specific meaning.

zaptheimpaler 2y ago
I don't think you understand what stealing means if you are unable to recognize that distributing free digital copies of paid books without agreed upon renumeration to the owners is stealing. It may or may not also be copyright infringement in a legal sense, but it sure is stealing in a common sense.

bilsbie 2y ago

Wait until you find out about libraries. Free books that humans are using to train the LLM’s in their heads.

bayindirh 2y ago
How many books a human can read, understand, tokenize, memorize in 15 minutes, 15 days, 15 months, 15 years? What's the rate of forgetting for the ingested material for human brain?
zaptheimpaler 2y ago
Libraries pay authors for their copies.

harshreality 2y ago

What do you think the market is for people who want to read books or articles condensed into lines of text suitable for model training? I think it's 0. Maybe a handful if pirate libraries of readable ebooks and papers didn't already exist, but even a handful of copyright violators wouldn't be a serious commercial threat to the publishers. So far, governments have been unable to do anything about pirate libraries. Complaining about AI datasets, poorly formatted for human consumption, seems misplaced. Criticizing this as "stealing" or "piracy" is a vote for a future where only big tech, or the major publishers themselves, can train models on good datasets. Nobody else will have the money or market power to license that much material.

zaptheimpaler 2y ago
The market for the entire contents of say OpenAI's entire codebase mangled and corrupted at parts might be 0 as well, doesn't make it not stealing. When NVIDIA's or Samsung's IP was leaked including binary firmware that is hard to read and understood by very few, that doesn't magically mean stealing and releasing it became legal.
- harshreality 2y ago
  It doesn't magically mean anything, but you're comparing a case where there's a plausible argument for fair use (no commercial impact from the training set sharing, not done for profit, transformative), against... what specific instance are you talking about with NV or Samsung? Theft of trade secrets? Samsung leaking their own IP to chatgpt, through stupidity, isn't illegal. This doesn't even affect the major AI companies that are doing training, does it? Don't you think Google, OpenAI, Anthropic all have their own datasets by now? If they wanted to use bulk content from pirate ebook and academic paper libraries, they would've already mirrored most of those sites a long time ago.