7 ms·

Most of the distribution for this is via torrents/magnet links and in person hard drive exchanges. I'd go look at some public trackers if you want a copy and do

Most of the distribution for this is via torrents/magnet links and in person hard drive exchanges. I'd go look at some public trackers if you want a copy and don't know someone that already has it. Do be aware that it does include copyrighted content so distribution is piracy.

Der_Einzige 3y ago
Almost all LLM training datasets include copyrighted content so almost all open source LLM distribution is piracy and almost all API based LLMs, including ChatGPT, are also piracy and copyright laundering. Also, most image-text dataset pairs contain far worse than that. You might want to check out LAION-5B and what stanford researchers have found in there. Technically, anyone who even touched that could in theory be in some serious, serious trouble. I find it quite remarkable that nothing has happened yet.
- littlestymaar 3y ago
  It's only piracy if it's private individual doing it, otherwise it's just “ask for forgiveness not for permission”-type Capitalism.
  - gosub100 3y ago
    It'll be some epic lawsuit like google-v-samsung that will get drawn out for a decade, awarded, and reduced, appealed, etc. where the only winners will be both party's lawyers.
    - littlestymaar 3y ago
      It's gonna be way worse than this: - OpenAI and others will just settle with MPAA, RIAA and the likes for a revenue stream (a single digit billion a year, likely) + some kind of control over what people can and cannot do with the AI + the access to the technology to produce their own content. - artists will see peanuts from the deal, and the big names are going to be able to stop doing any kind of business with artists which are just expenses in their eyes. They will have been replaced by machines that where trained using their art with no compensation whatsoever. IP is already predatory capitalism, AI will definitely be weaponized against the workers by the owners of the means of “production”.
- beeboobaa 3y ago
  Turns out you can ignore copyright law if your company has enough money.
- vineyardmike 3y ago
  The courts (in the US) have not found LLM model weights to be piracy, nor the outputs, but it’s really surprising that LAION was used for so long consider the content you allude to.
  - Filligree 3y ago
    LAION is essentially a list of every image on the public internet. It was filtered, of course, but do you really expect perfection? It's impossible to create such a list while evading all such material.
    - vineyardmike 3y ago
      There exists databases of “the hash of problematic photos” (CSAM), so it seems trivial to search your billions of photos against them before training an AI. You can’t catch everything, but this seems like an obvious miss considering the explicitly tried to scrape pornography. These hashes is exactly how researchers later discovered this content, so it’s clearly not hard.
      - duskwuff 3y ago
        The Stanford researchers also found a substantial number of CSAM images in the LAION-5B dataset which were not recognized by PhotoDNA, probably because the images in question were not in wide distribution prior to their inclusion in LAION. Full paper: https://stacks.stanford.edu/file/druid:kh752sm9123/ml_training_data_csam_report-2023-12-23.pdf https://stacks.stanford.edu/file/druid:kh752sm9123/ml_traini...
      - SEGyges 3y ago
        You are uploading 5 billion examples of <something>. You cannot filter it manually, of course, because there are five billion of it. Given that it is the year 2024, how hard is it to be positive that a well-resourced team at Stanford in 2029 will not have better methods of identifying and filtering your data, or a better reference dataset to filter it against, than you do presently? It is a pretty hard problem.
- visarga 3y ago
  > almost all open source LLM distribution is piracy and almost all API based LLMs, including ChatGPT, are also piracy and copyright laundering That's an amplification of copyright, original expression is protected, but not the ideas themselves, those are free. And don't forget when we actually get to use these models we feed them questions, data, we give corrections - so they are not simply replicating the training set, they learn and do new things with new inputs. In fact if you think deeply about it, it is silly to accuse AI of copyright violation. Copying the actual book or article is much much faster and cheaper, and exact. Why would I pay a LLM provider to generate it for me from the title and starting phrase? If I already have part of the article, do I still need to generate it with AI? it's silly. LLM regurgitation are basically attacks with special key, entrapments. They don't happen in normal use.
- doctorpangloss 3y ago
  > I find it quite remarkable that nothing has happened yet. While I don't think it's because you're wrong, per se, it's just that none of this drama really matters.
- Workaccount2 3y ago
  Models are not information archives. The size of the final model is orders of magnitude smaller than the size of the training data. Somehow people are just not able to get this through their heads. Stable diffusion is like 12GB or something and you have people convinced it's a tool that is cutting and pasting copyrighted works from an enormous image archive.
  - 7moritz7 3y ago
    Stable Diffusion 1.5 is 1.5 to 6 GB depending on the finetune and trained on like 5 billion images
  - feoren 3y ago
    > The size of the final model is orders of magnitude smaller than the size of the training data. Good to know I can avoid copyright on a book just by zipping it up!
    - Workaccount2 3y ago
      No, you can't. But you can by reading the book, writing down the general gist of it (even including some passages), and then storing that. LLM's are not compressing petabytes of information down to a few gigabytes.

Der_Einzige 3y ago

Almost all LLM training datasets include copyrighted content so almost all open source LLM distribution is piracy and almost all API based LLMs, including ChatGPT, are also piracy and copyright laundering. Also, most image-text dataset pairs contain far worse than that. You might want to check out LAION-5B and what stanford researchers have found in there. Technically, anyone who even touched that could in theory be in some serious, serious trouble. I find it quite remarkable that nothing has happened yet.

littlestymaar 3y ago
It's only piracy if it's private individual doing it, otherwise it's just “ask for forgiveness not for permission”-type Capitalism.
- gosub100 3y ago
  It'll be some epic lawsuit like google-v-samsung that will get drawn out for a decade, awarded, and reduced, appealed, etc. where the only winners will be both party's lawyers.
  - littlestymaar 3y ago
    It's gonna be way worse than this: - OpenAI and others will just settle with MPAA, RIAA and the likes for a revenue stream (a single digit billion a year, likely) + some kind of control over what people can and cannot do with the AI + the access to the technology to produce their own content. - artists will see peanuts from the deal, and the big names are going to be able to stop doing any kind of business with artists which are just expenses in their eyes. They will have been replaced by machines that where trained using their art with no compensation whatsoever. IP is already predatory capitalism, AI will definitely be weaponized against the workers by the owners of the means of “production”.
beeboobaa 3y ago
Turns out you can ignore copyright law if your company has enough money.
vineyardmike 3y ago
The courts (in the US) have not found LLM model weights to be piracy, nor the outputs, but it’s really surprising that LAION was used for so long consider the content you allude to.
- Filligree 3y ago
  LAION is essentially a list of every image on the public internet. It was filtered, of course, but do you really expect perfection? It's impossible to create such a list while evading all such material.
  - vineyardmike 3y ago
    There exists databases of “the hash of problematic photos” (CSAM), so it seems trivial to search your billions of photos against them before training an AI. You can’t catch everything, but this seems like an obvious miss considering the explicitly tried to scrape pornography. These hashes is exactly how researchers later discovered this content, so it’s clearly not hard.
    - duskwuff 3y ago
      The Stanford researchers also found a substantial number of CSAM images in the LAION-5B dataset which were not recognized by PhotoDNA, probably because the images in question were not in wide distribution prior to their inclusion in LAION. Full paper: https://stacks.stanford.edu/file/druid:kh752sm9123/ml_training_data_csam_report-2023-12-23.pdf https://stacks.stanford.edu/file/druid:kh752sm9123/ml_traini...
    - SEGyges 3y ago
      You are uploading 5 billion examples of <something>. You cannot filter it manually, of course, because there are five billion of it. Given that it is the year 2024, how hard is it to be positive that a well-resourced team at Stanford in 2029 will not have better methods of identifying and filtering your data, or a better reference dataset to filter it against, than you do presently? It is a pretty hard problem.
visarga 3y ago
> almost all open source LLM distribution is piracy and almost all API based LLMs, including ChatGPT, are also piracy and copyright laundering That's an amplification of copyright, original expression is protected, but not the ideas themselves, those are free. And don't forget when we actually get to use these models we feed them questions, data, we give corrections - so they are not simply replicating the training set, they learn and do new things with new inputs. In fact if you think deeply about it, it is silly to accuse AI of copyright violation. Copying the actual book or article is much much faster and cheaper, and exact. Why would I pay a LLM provider to generate it for me from the title and starting phrase? If I already have part of the article, do I still need to generate it with AI? it's silly. LLM regurgitation are basically attacks with special key, entrapments. They don't happen in normal use.
doctorpangloss 3y ago
> I find it quite remarkable that nothing has happened yet. While I don't think it's because you're wrong, per se, it's just that none of this drama really matters.
Workaccount2 3y ago
Models are not information archives. The size of the final model is orders of magnitude smaller than the size of the training data. Somehow people are just not able to get this through their heads. Stable diffusion is like 12GB or something and you have people convinced it's a tool that is cutting and pasting copyrighted works from an enormous image archive.
- 7moritz7 3y ago
  Stable Diffusion 1.5 is 1.5 to 6 GB depending on the finetune and trained on like 5 billion images
- feoren 3y ago
  > The size of the final model is orders of magnitude smaller than the size of the training data. Good to know I can avoid copyright on a book just by zipping it up!
  - Workaccount2 3y ago
    No, you can't. But you can by reading the book, writing down the general gist of it (even including some passages), and then storing that. LLM's are not compressing petabytes of information down to a few gigabytes.