5 ms·

Author here. And by author I mean I created books3 (the books component of The Pile) while everyone else did the hard work of actually writing the paper, ha. St

Author here. And by author I mean I created books3 (the books component of The Pile) while everyone else did the hard work of actually writing the paper, ha. Stella and Leo Gao in particular did so much wonderful work on the paper, though it couldn’t have happened without everyone’s contributions. As far as I know, this was the first academic contribution from a discord collaboration to ML. Back then discord was barely used for ML at all, though nowadays of course the largest discord in the world is midjourney. There were a bunch of interesting stories from those days. We almost didn’t release at all (or at least the books component) because of fear of copyright backlash. Turns out no one cared, and then suddenly today the world cares a great deal. As a side note, I’ll be participating in a legal action against Meta for the purpose of making ML models uncopyrightable: https://twitter.com/theshawwn/status/1641804013791215619?s=61&t=jQbmCk1JqL7depzFWJNuPA https://twitter.com/theshawwn/status/1641804013791215619?s=6.... They DMCA’ed one of my repos distributing LLaMA, so we fought back and challenged the idea that weights can be copyrighted at all. This seems like the best outcome for hackers and individual researchers, for a few reasons. It’s also one of the most ethical outcomes; since ~no one trains on data that they own, they shouldn’t own the resulting model. One last thing. The Pile would’ve been far less relevant without the wonderful assistance of The Eye, a group of people who archive all kinds of things. They’ve hosted the datasets for years now. And although it seems strange to say that dataset hosting could make or break The Pile, back then there was nobody else willing to host us. https://the-eye.eu/ https://the-eye.eu/

sfriedr 3y ago
Could you share more about copyright? For example, aren't you worried that now, with all kinds of lawsuits happening [1] and copyright issues that were found in existing datasets [2], that you might get threatening letters from a lawyer some day? I'm the author of [3] where we introduced one of the first natural-language datasets that test graduate mathematics for LLMs, but some of the prompts we took from a copyrighted book and therefore thought about excluding them. Having them in the public dataset would be really nice though, hence I'm keen about your experience. I'd also be keen to hear how your challenge against the DMCA on sharing LLaMA's weights goes? [1] https://www.theguardian.com/books/2023/jul/05/authors-file-a-lawsuit-against-openai-for-unlawfully-ingesting-their-books https://www.theguardian.com/books/2023/jul/05/authors-file-a... [2] https://arxiv.org/abs/2105.05241 https://arxiv.org/abs/2105.05241 [3] https://arxiv.org/abs/2301.13867 https://arxiv.org/abs/2301.13867
- Der_Einzige 3y ago
  Getting sued is straight up a good thing for most peoples careers in tech. Haven't you watched silicon valley?
- sillysaurusx 3y ago
  I think a lot of hackers shy away from doing impactful work because of fear. Sometimes those fears are justified, but it's remarkable how often things that seem like a big deal turn out not to matter. My advice for ambitious devs would be to do what seems interesting, and don't worry too much about threatening letters. Usually the worst thing that happens is that you agree to stop doing whatever generated the threat. Personally, I'm not worried. It would be a damn shame if academics come under fire merely for trying to operate on the cutting edge of science. None of us were trying to make money; we just wanted to make something interesting. > I'd also be keen to hear how your challenge against the DMCA on sharing LLaMA's weights goes? Thanks! I think we might be putting up a website for it soon, if only to explain ourselves. In the meantime – I hate this phrase, since I don't want followers – the only way to keep informed is to follow my Twitter, and perhaps keep an eye on my HN comments. You'll probably hear about it either way though, since it's a groundbreaking case. No one has tested the copyrightability of ML models before.
  - jacquesm 3y ago
    What exactly is it that you claim copyright over? Are you sure that you have standing to bring that suit?
    - sillysaurusx 3y ago
      It’s the other way around — Meta DMCA’d llama-dl, my github repo, claiming they control copyright of llama. Our assertion is that ML weights are uncopyrightable, much like a phone book - training a model on the same dataset in the same way usually gives more or less the same model, even if the weights are completely different each time. I can send you the draft we’ve prepared if you’re interested — drop me an email. But I’ll probably set up a site for this, if only to clear up our motives and expected outcomes.
hedgehog 3y ago
I understand that LLMs to date have mostly been trained on a wide variety of copyright-encumbered data but in other domains (computer vision for example) the tradeoffs are different and in practice many models are trained on private / unencumbered data. If those weights are not protected by copyright then my concern is it will be hard to sufficiently protect them via license agreement and it will become yet another factor favoring the SaaSification of everything in tech.
- idiotsecant 3y ago
  Is it useful to protect weights with copyright? What if I download your weights and retrain them for 5 seconds, changing each weight .0000001%? How much change is a new product? What if I change a single weight?
  - hedgehog 3y ago
    Like the parallel scenarios of taking a book and changing a few words, slapping a new logo on someone else's app, or stylizing a photo with a filter, those are questions that will be answered in court if people can't come to an agreement on their own.
- sillysaurusx 3y ago
  This is true, and it's why I hesitated to file legal action. My goal was to benefit hackers. If the outcome causes problems for people who are just trying to share their work, I'd be upset. Ultimately what convinced me to proceed is that there are immense forces pressuring ML models to become SaaS companies. It's very difficult to offer an ML model for extended periods without being a company. E.g. https://6b.eleuther.ai/ https://6b.eleuther.ai/ is down. Eleuther failing illustrates just how hard it is –– we were all working as hard as we could to design something that would last a long time, and a long time turned out to be two short years. Contrast that with other kinds of hacking (e.g. webdev, gamedev, hardware...) where the end result lasts basically forever. So if ML models aren't copyrightable, I think it'll hurt companies a lot more than individuals. In fact the goal is the other way around: to protect individuals. All I did was publish Facebook's own GPL download script to github, and it got DMCA'd. If we don't push back on that kind of behavior now, companies will get used to the idea that they control "their" model –– even when their model is anything but theirs.
jacquesm 3y ago
> It’s also one of the most ethical outcomes; since ~no one trains on data that they own, they shouldn’t own the resulting model. In my opinion the most ethical outcome would be that they are on the hook for the cumulative cost of the copyright they violated. That way authors would come out ahead instead of having their rights trashed 'because it's too late anyway'.
- cornel_io 3y ago
  Whether or not training on publicly available data counts as a copyright violation is still completely up in the air legally, and clearly a lot of lawyers at all of the top tech companies think they're going to end up in the clear under fair use. At some point this stuff will have to get tested by making its way up the appeals stack in the US, and IMO there is only a minuscule chance that will result in Google, MS, and Meta getting slapped with anything more than a token fine (my bet is it won't even be that), let alone paying every person who ever wrote anything that was used in these datasets for copyright violations, which would basically be everyone.
  - jacquesm 3y ago
    There are more courts than just the US ones.
    - lucubratory 3y ago
      Yes, there are other courts than the US ones, and generally the law there is significantly more favourable to TDM with regards to copyright, with the exception of the PRC. Examples: Japan: Article 30-4 of the Japanese Copyright Act. No special action on the part of companies is necessary for compliance. All models are legal so long as their output is legal. The UK: s.29A of the Copyright, Designs and Patents Act 1988 (CDPA). Models must be trained by non-profit research institutes, and can then be used by anyone (including for profit entities); similar to the Stable Diffusion model. The EU: Articles 3 & 4 of the Directive on Copyright in the Digital Single Market (CDSM). There are no restrictions on non-profit TDM, same as the UK. For-profit TDM is exempted from copyright so long as the data harvesting process respects an "opt-out" process, where specific contractual forms/disclosures of opting out of inclusion in the training data are respected. Singapore: Articles 243 & 244 of the Copyright Act. No special action on the part of companies is necessary for compliance. All models are legal so long as their output is legal.
archivist0 3y ago
> One last thing. The Pile would’ve been far less relevant without the wonderful assistance of The Eye, a group of people who archive all kinds of things. They’ve hosted the datasets for years now. And although it seems strange to say that dataset hosting could make or break The Pile, back then there was nobody else willing to host us. https://the-eye.eu/ https://the-eye.eu/ I'm afraid to say... the-eye no longer hosts the pile as of today due to legal threats above the likes of DMCA. Though I believe it's still available via its original torrent and on at. > https://academictorrents.com/details/0d366035664fdf51cfbe9f733953ba325776e667 https://academictorrents.com/details/0d366035664fdf51cfbe9f7...
- sfriedr 3y ago
  Of this is true, it would be something close of an insane situation: One of the largest datasets, that the largest companies are using to train their models (probably; many of the best LLMs have technical reports that raise more questions rather than answer them) being forced to live an obscure existance on torrents. From a scientific point of view this is very problematic because few safeguards exist that guarantee that the dataset is not tampered with (as is the case if you'd upload it to Zenodo, which providea some guarantee of immutability). How about trying to upload the Pile to Zenodo? Only half-joking :D
- koheripbal 3y ago
  I'm more interested in The Pile V2 which seems to have gone underground...
andy99 3y ago
Hi Shawn, re your side note, I disagree with you that we'd be better off if weights couldn't be copyrighted - basically because copyright gives options like GPL that can keep models open, otherwise we're just going to see everything good disappear behind trade secret. That said I fully support your "civil disobedience" in sharing the weights. I don't expect you to agree, but take a look at something I just wrote about this yesterday: http://marble.onl/posts/model_weight_copyrights.html http://marble.onl/posts/model_weight_copyrights.html . I'm happy to chat about it if you're interested.
- sillysaurusx 3y ago
  It might seem like I’m entrenched in my position, but it’s quite the opposite — the only reason I’m doing this is because I really believe it’s the best outcome for devs in the long run. I’m open to changing my mind and pulling the plug in everything. I’ll read over your essay and give it some thought. There are a bunch of subtle aspects to consider; I’ve been thinking it over for about four months now and still haven’t covered all the territory yet. It feels like this may be one of the most important decisions going forward — both from an intellectual property point of view, and an individual rights perspective. E.g. you mention that it’s civil disobedience to share the weights, but it feels like if someone is claiming to do open science (LLaMA), sharing the research materials is the minimum requirement. Plus look how it’s benefited them; they’ve captured most of the open source LLM mindshare. So it seems likely that this will lead to more open source work in the long run, not less. Feel free to chat! You can DM me on Twitter or email me. I’ve been in the hospital with my wife for 7 weeks, with two to go, so I’ve been a bit less responsive than I usually am.
  - scoot 3y ago
    [flagged]
- js8 3y ago
  > otherwise we're just going to see everything good disappear behind trade secret We will see that anyway. All the code I work on commercially is copyrighted and yet a trade secret. Existence of copyright (with the exception of copyleft, but that's subversion) didn't help software to be open sourced. IMHO allowing models to be copyrighted is basically 18th century enclosures again.
lukemerrick 3y ago
Related to the idea of "no one trains on data they own, they shouldn't own the resulting model": since big public datasets like The Pile have CC-SA items in them, is anyone considering bringing the argument that model weights are derivative work that must be "shared alike"?
- koheripbal 3y ago
  By that token, my brain is a derivative work of all the copyrighted works I've consumed
koheripbal 3y ago
What ever happened with The Pile V2? I spent a couple hours searching for it, but the Eye is impossible to navigate and people on the discord generally invite noobs like myself.
moffkalast 3y ago
> They DMCA’ed one of my repos distributing LLaMA Boy they'll be mad once they learn about Huggingface distributing thousands of LLama fine tunes with full weights.
- redox99 3y ago
  Weights being copyrightable is already a questionable thing. Derivatives (like finetunes) is even more questionable.

sfriedr 3y ago

Could you share more about copyright? For example, aren't you worried that now, with all kinds of lawsuits happening [1] and copyright issues that were found in existing datasets [2], that you might get threatening letters from a lawyer some day? I'm the author of [3] where we introduced one of the first natural-language datasets that test graduate mathematics for LLMs, but some of the prompts we took from a copyrighted book and therefore thought about excluding them. Having them in the public dataset would be really nice though, hence I'm keen about your experience. I'd also be keen to hear how your challenge against the DMCA on sharing LLaMA's weights goes? [1] https://www.theguardian.com/books/2023/jul/05/authors-file-a-lawsuit-against-openai-for-unlawfully-ingesting-their-books https://www.theguardian.com/books/2023/jul/05/authors-file-a... [2] https://arxiv.org/abs/2105.05241 https://arxiv.org/abs/2105.05241 [3] https://arxiv.org/abs/2301.13867 https://arxiv.org/abs/2301.13867

Der_Einzige 3y ago
Getting sued is straight up a good thing for most peoples careers in tech. Haven't you watched silicon valley?
sillysaurusx 3y ago
I think a lot of hackers shy away from doing impactful work because of fear. Sometimes those fears are justified, but it's remarkable how often things that seem like a big deal turn out not to matter. My advice for ambitious devs would be to do what seems interesting, and don't worry too much about threatening letters. Usually the worst thing that happens is that you agree to stop doing whatever generated the threat. Personally, I'm not worried. It would be a damn shame if academics come under fire merely for trying to operate on the cutting edge of science. None of us were trying to make money; we just wanted to make something interesting. > I'd also be keen to hear how your challenge against the DMCA on sharing LLaMA's weights goes? Thanks! I think we might be putting up a website for it soon, if only to explain ourselves. In the meantime – I hate this phrase, since I don't want followers – the only way to keep informed is to follow my Twitter, and perhaps keep an eye on my HN comments. You'll probably hear about it either way though, since it's a groundbreaking case. No one has tested the copyrightability of ML models before.
- jacquesm 3y ago
  What exactly is it that you claim copyright over? Are you sure that you have standing to bring that suit?
  - sillysaurusx 3y ago
    It’s the other way around — Meta DMCA’d llama-dl, my github repo, claiming they control copyright of llama. Our assertion is that ML weights are uncopyrightable, much like a phone book - training a model on the same dataset in the same way usually gives more or less the same model, even if the weights are completely different each time. I can send you the draft we’ve prepared if you’re interested — drop me an email. But I’ll probably set up a site for this, if only to clear up our motives and expected outcomes.

hedgehog 3y ago

I understand that LLMs to date have mostly been trained on a wide variety of copyright-encumbered data but in other domains (computer vision for example) the tradeoffs are different and in practice many models are trained on private / unencumbered data. If those weights are not protected by copyright then my concern is it will be hard to sufficiently protect them via license agreement and it will become yet another factor favoring the SaaSification of everything in tech.

idiotsecant 3y ago
Is it useful to protect weights with copyright? What if I download your weights and retrain them for 5 seconds, changing each weight .0000001%? How much change is a new product? What if I change a single weight?
- hedgehog 3y ago
  Like the parallel scenarios of taking a book and changing a few words, slapping a new logo on someone else's app, or stylizing a photo with a filter, those are questions that will be answered in court if people can't come to an agreement on their own.
sillysaurusx 3y ago
This is true, and it's why I hesitated to file legal action. My goal was to benefit hackers. If the outcome causes problems for people who are just trying to share their work, I'd be upset. Ultimately what convinced me to proceed is that there are immense forces pressuring ML models to become SaaS companies. It's very difficult to offer an ML model for extended periods without being a company. E.g. https://6b.eleuther.ai/ https://6b.eleuther.ai/ is down. Eleuther failing illustrates just how hard it is –– we were all working as hard as we could to design something that would last a long time, and a long time turned out to be two short years. Contrast that with other kinds of hacking (e.g. webdev, gamedev, hardware...) where the end result lasts basically forever. So if ML models aren't copyrightable, I think it'll hurt companies a lot more than individuals. In fact the goal is the other way around: to protect individuals. All I did was publish Facebook's own GPL download script to github, and it got DMCA'd. If we don't push back on that kind of behavior now, companies will get used to the idea that they control "their" model –– even when their model is anything but theirs.

jacquesm 3y ago

> It’s also one of the most ethical outcomes; since ~no one trains on data that they own, they shouldn’t own the resulting model. In my opinion the most ethical outcome would be that they are on the hook for the cumulative cost of the copyright they violated. That way authors would come out ahead instead of having their rights trashed 'because it's too late anyway'.

cornel_io 3y ago
Whether or not training on publicly available data counts as a copyright violation is still completely up in the air legally, and clearly a lot of lawyers at all of the top tech companies think they're going to end up in the clear under fair use. At some point this stuff will have to get tested by making its way up the appeals stack in the US, and IMO there is only a minuscule chance that will result in Google, MS, and Meta getting slapped with anything more than a token fine (my bet is it won't even be that), let alone paying every person who ever wrote anything that was used in these datasets for copyright violations, which would basically be everyone.
- jacquesm 3y ago
  There are more courts than just the US ones.
  - lucubratory 3y ago
    Yes, there are other courts than the US ones, and generally the law there is significantly more favourable to TDM with regards to copyright, with the exception of the PRC. Examples: Japan: Article 30-4 of the Japanese Copyright Act. No special action on the part of companies is necessary for compliance. All models are legal so long as their output is legal. The UK: s.29A of the Copyright, Designs and Patents Act 1988 (CDPA). Models must be trained by non-profit research institutes, and can then be used by anyone (including for profit entities); similar to the Stable Diffusion model. The EU: Articles 3 & 4 of the Directive on Copyright in the Digital Single Market (CDSM). There are no restrictions on non-profit TDM, same as the UK. For-profit TDM is exempted from copyright so long as the data harvesting process respects an "opt-out" process, where specific contractual forms/disclosures of opting out of inclusion in the training data are respected. Singapore: Articles 243 & 244 of the Copyright Act. No special action on the part of companies is necessary for compliance. All models are legal so long as their output is legal.

archivist0 3y ago

> One last thing. The Pile would’ve been far less relevant without the wonderful assistance of The Eye, a group of people who archive all kinds of things. They’ve hosted the datasets for years now. And although it seems strange to say that dataset hosting could make or break The Pile, back then there was nobody else willing to host us. https://the-eye.eu/ https://the-eye.eu/ I'm afraid to say... the-eye no longer hosts the pile as of today due to legal threats above the likes of DMCA. Though I believe it's still available via its original torrent and on at. > https://academictorrents.com/details/0d366035664fdf51cfbe9f733953ba325776e667 https://academictorrents.com/details/0d366035664fdf51cfbe9f7...

sfriedr 3y ago
Of this is true, it would be something close of an insane situation: One of the largest datasets, that the largest companies are using to train their models (probably; many of the best LLMs have technical reports that raise more questions rather than answer them) being forced to live an obscure existance on torrents. From a scientific point of view this is very problematic because few safeguards exist that guarantee that the dataset is not tampered with (as is the case if you'd upload it to Zenodo, which providea some guarantee of immutability). How about trying to upload the Pile to Zenodo? Only half-joking :D
koheripbal 3y ago
I'm more interested in The Pile V2 which seems to have gone underground...

andy99 3y ago

Hi Shawn, re your side note, I disagree with you that we'd be better off if weights couldn't be copyrighted - basically because copyright gives options like GPL that can keep models open, otherwise we're just going to see everything good disappear behind trade secret. That said I fully support your "civil disobedience" in sharing the weights. I don't expect you to agree, but take a look at something I just wrote about this yesterday: http://marble.onl/posts/model_weight_copyrights.html http://marble.onl/posts/model_weight_copyrights.html . I'm happy to chat about it if you're interested.

sillysaurusx 3y ago
It might seem like I’m entrenched in my position, but it’s quite the opposite — the only reason I’m doing this is because I really believe it’s the best outcome for devs in the long run. I’m open to changing my mind and pulling the plug in everything. I’ll read over your essay and give it some thought. There are a bunch of subtle aspects to consider; I’ve been thinking it over for about four months now and still haven’t covered all the territory yet. It feels like this may be one of the most important decisions going forward — both from an intellectual property point of view, and an individual rights perspective. E.g. you mention that it’s civil disobedience to share the weights, but it feels like if someone is claiming to do open science (LLaMA), sharing the research materials is the minimum requirement. Plus look how it’s benefited them; they’ve captured most of the open source LLM mindshare. So it seems likely that this will lead to more open source work in the long run, not less. Feel free to chat! You can DM me on Twitter or email me. I’ve been in the hospital with my wife for 7 weeks, with two to go, so I’ve been a bit less responsive than I usually am.
- scoot 3y ago
  [flagged]
js8 3y ago
> otherwise we're just going to see everything good disappear behind trade secret We will see that anyway. All the code I work on commercially is copyrighted and yet a trade secret. Existence of copyright (with the exception of copyleft, but that's subversion) didn't help software to be open sourced. IMHO allowing models to be copyrighted is basically 18th century enclosures again.

lukemerrick 3y ago

Related to the idea of "no one trains on data they own, they shouldn't own the resulting model": since big public datasets like The Pile have CC-SA items in them, is anyone considering bringing the argument that model weights are derivative work that must be "shared alike"?

koheripbal 3y ago
By that token, my brain is a derivative work of all the copyrighted works I've consumed

koheripbal 3y ago

What ever happened with The Pile V2? I spent a couple hours searching for it, but the Eye is impossible to navigate and people on the discord generally invite noobs like myself.

moffkalast 3y ago

> They DMCA’ed one of my repos distributing LLaMA Boy they'll be mad once they learn about Huggingface distributing thousands of LLama fine tunes with full weights.

redox99 3y ago
Weights being copyrightable is already a questionable thing. Derivatives (like finetunes) is even more questionable.