5 ms·

Linked library has a DCMA takedown. Host this stuff outside of US corporate control.

by toastal 4y ago

sliken 4y ago
Or just post a magnet link to a torrent.
- knodi123 4y ago
  Downloading alpaca weights actually does use a torrent now! But it seems a little.... leechy... to use a fire-and-forget CLI torrent client to grab a dozens of gigabytes file and then quit.
  - capableweb 4y ago
    Best would be to let the seeding happen for a while, until it at least reaches 0.5 ratio or something like that. But downloading over torrent is still better than over HTTP, it'll still seed the pieces (unless the downloader turned that off fully, that'd be an asshole move) as it download the missing pieces.
shaky-carrousel 4y ago
If the model weights are able to be copyrighted, then they are also able to be a derivative work of something copyrighted, as the model is based on copyrighted works. So either the weights are infringing copyright, which means they don't belong to Meta, or the weights can't be copyrighted. Either way, that DMCA is worthless.
- demoride 4y ago
  Your philosophical argument is interesting, but what the op was saying was one of the linked repos used in this repo is inaccessible due to DMCA: https://github.com/shawwn/llama-dl https://github.com/shawwn/llama-dl So while what you say may be true the DMCA seems to have worth for these orgs because they can get code removed by the host, who is uninterested in litigating, and the repo owner likely is even less capable of litigating the DMCA. Unfortunately as a tool of fear and legal gridlock DMCA has shown itself to be very useful to those with ill intent.
- halfeatenscone 4y ago
  I don't think this follows. Calling it a derivative work already feels like a stretch, but even granting that framing, the use is clearly transformative and therefore likely to be considered fair use in the US. I actually wrote a Wikipedia article on the intersection of copyright law and deep learning models the other day (https://en.wikipedia.org/wiki/Artificial_intelligence_and_copyright https://en.wikipedia.org/wiki/Artificial_intelligence_and_co...). I was hoping to include a section on the copyrightability of model weights, but was sadly able to find 0 coverage in reliable sources.
  - nl 4y ago
    This should include https://arxiv.org/abs/2303.15715 https://arxiv.org/abs/2303.15715
  - kmeisthax 4y ago
    So, while the poster you replied to has the wrong reasoning, they got to the right conclusion. Mostly. Let's start with a non-AI example: compilers. If I have the source code for Linux, I can compile it, but I don't own the kernel binaries I made. This is because the compiler is a purely mechanical process, not a tool of human creativity. Copyright in the US attaches to creativity from human authors. So the source code would be the creative work, not the binaries. We don't normally talk about this because ownership over the source code still flows through to the binaries. Your permission to copy that Linux binary is downstream of Linus having granted you permission to do so under the GPL. If you had instead copied, say, the NT kernel, you would be infringing the copyright on the NT kernel source code by distributing binaries of it. So now let's go to AI land. You've collected a bunch of training data and dumped it into a linear algebra blender. That's like compiling source code: the ML trainer program adds no creativity or authorship, so you haven't gained any ownership over the data. Remember: this training data is scraped off the Internet from other people's work. Fair use merely makes it non-infringing to do this, it does not mean you own the result. There are two avenues by which Meta could still get US copyright over the language model: - They could make a model with their own training data that they made, and use their ownership over the training data to get ownership over the model. - They could assert ownership over the compilation of training data. Compilation ownership is kind of weird. Basically, in the US, you can make a compilation of other people's work and own solely that. Like, say, a "Top 10 Songs I Like" playlist[0]. But even then the creativity and authorship rules still apply. These models are not being trained by having humans manually select specific works that would do well in the model. They scrape the Internet and train on everything[1]. In fact, they usually don't even use their own scrapes; they use Common Crawl, LAION-5B, and/or The Pile. Whether or not any of this is right would require someone to actually share LLaMA, get sued by Facebook, and then assert this legal theory. And hope that Facebook does not assert any other legal claims, such as misappropriation of trade secrets, which might actually stick. [0] Or in a particularly egregious example, someone copyrighting their Magic: The Gathering deck in protest of this nonsense. [1] Stable Diffusion at least uses an "aesthetics score", but AFAIK that's generated by an AI so also not copyrightable.
- hbn 4y ago
  I'm not sure the logic follows. Music can be copyrighted, and it can also be made from samples from other music that's copyrighted. But sampling still happens without infringing copyright. I'd say the small handful of bits flipped in a model from training on some text, or piece of code, or an image is even less copyrighted information brought over than a music sample or borrowed/referenced melody.
- nl 4y ago
  I'd suggest all the internet lawyers on this thread read "Foundation models and Fair Use": https://arxiv.org/abs/2303.15715 https://arxiv.org/abs/2303.15715 TL;DR: There's a good fair use argument but it isn't a given that fair use always applies.
- am44jnsf 4y ago
  all work is derivative.

Linked library has a DCMA takedown. Host this stuff outside of US corporate control.

sliken 4y ago

Or just post a magnet link to a torrent.

knodi123 4y ago
Downloading alpaca weights actually does use a torrent now! But it seems a little.... leechy... to use a fire-and-forget CLI torrent client to grab a dozens of gigabytes file and then quit.
- capableweb 4y ago
  Best would be to let the seeding happen for a while, until it at least reaches 0.5 ratio or something like that. But downloading over torrent is still better than over HTTP, it'll still seed the pieces (unless the downloader turned that off fully, that'd be an asshole move) as it download the missing pieces.

shaky-carrousel 4y ago

If the model weights are able to be copyrighted, then they are also able to be a derivative work of something copyrighted, as the model is based on copyrighted works. So either the weights are infringing copyright, which means they don't belong to Meta, or the weights can't be copyrighted. Either way, that DMCA is worthless.

demoride 4y ago
Your philosophical argument is interesting, but what the op was saying was one of the linked repos used in this repo is inaccessible due to DMCA: https://github.com/shawwn/llama-dl https://github.com/shawwn/llama-dl So while what you say may be true the DMCA seems to have worth for these orgs because they can get code removed by the host, who is uninterested in litigating, and the repo owner likely is even less capable of litigating the DMCA. Unfortunately as a tool of fear and legal gridlock DMCA has shown itself to be very useful to those with ill intent.
halfeatenscone 4y ago
I don't think this follows. Calling it a derivative work already feels like a stretch, but even granting that framing, the use is clearly transformative and therefore likely to be considered fair use in the US. I actually wrote a Wikipedia article on the intersection of copyright law and deep learning models the other day (https://en.wikipedia.org/wiki/Artificial_intelligence_and_copyright https://en.wikipedia.org/wiki/Artificial_intelligence_and_co...). I was hoping to include a section on the copyrightability of model weights, but was sadly able to find 0 coverage in reliable sources.
- nl 4y ago
  This should include https://arxiv.org/abs/2303.15715 https://arxiv.org/abs/2303.15715
- kmeisthax 4y ago
  So, while the poster you replied to has the wrong reasoning, they got to the right conclusion. Mostly. Let's start with a non-AI example: compilers. If I have the source code for Linux, I can compile it, but I don't own the kernel binaries I made. This is because the compiler is a purely mechanical process, not a tool of human creativity. Copyright in the US attaches to creativity from human authors. So the source code would be the creative work, not the binaries. We don't normally talk about this because ownership over the source code still flows through to the binaries. Your permission to copy that Linux binary is downstream of Linus having granted you permission to do so under the GPL. If you had instead copied, say, the NT kernel, you would be infringing the copyright on the NT kernel source code by distributing binaries of it. So now let's go to AI land. You've collected a bunch of training data and dumped it into a linear algebra blender. That's like compiling source code: the ML trainer program adds no creativity or authorship, so you haven't gained any ownership over the data. Remember: this training data is scraped off the Internet from other people's work. Fair use merely makes it non-infringing to do this, it does not mean you own the result. There are two avenues by which Meta could still get US copyright over the language model: - They could make a model with their own training data that they made, and use their ownership over the training data to get ownership over the model. - They could assert ownership over the compilation of training data. Compilation ownership is kind of weird. Basically, in the US, you can make a compilation of other people's work and own solely that. Like, say, a "Top 10 Songs I Like" playlist[0]. But even then the creativity and authorship rules still apply. These models are not being trained by having humans manually select specific works that would do well in the model. They scrape the Internet and train on everything[1]. In fact, they usually don't even use their own scrapes; they use Common Crawl, LAION-5B, and/or The Pile. Whether or not any of this is right would require someone to actually share LLaMA, get sued by Facebook, and then assert this legal theory. And hope that Facebook does not assert any other legal claims, such as misappropriation of trade secrets, which might actually stick. [0] Or in a particularly egregious example, someone copyrighting their Magic: The Gathering deck in protest of this nonsense. [1] Stable Diffusion at least uses an "aesthetics score", but AFAIK that's generated by an AI so also not copyrightable.
hbn 4y ago
I'm not sure the logic follows. Music can be copyrighted, and it can also be made from samples from other music that's copyrighted. But sampling still happens without infringing copyright. I'd say the small handful of bits flipped in a model from training on some text, or piece of code, or an image is even less copyrighted information brought over than a music sample or borrowed/referenced melody.
nl 4y ago
I'd suggest all the internet lawyers on this thread read "Foundation models and Fair Use": https://arxiv.org/abs/2303.15715 https://arxiv.org/abs/2303.15715 TL;DR: There's a good fair use argument but it isn't a given that fair use always applies.
am44jnsf 4y ago
all work is derivative.