3 ms·

You can publish these models in all jurisdictions, including ones that do care about IP law. There's no rulings saying that models trained on datasets of images

You can publish these models in all jurisdictions, including ones that do care about IP law. There's no rulings saying that models trained on datasets of images are direct derivatives of the original images (in a way that's copyright-violating), let alone whatever is produced using these models. Now, this isn't settled law by any means, but it feels like people give too much credence to all the theft accusations by assuming that it's some slam dunk case that can make all generative AI suddenly vanish.

kmeisthax 3y ago
I used to agree with this, but there's been research coming out of Google that has altered my opinion. Specifically, Google's gotten rather good at making AI spit out unaltered training set data[0]. This is only possible if the AI is remembering large portions of the original trained-on works, which would make the weights infringing. [0] In the most egregious case, they found that just asking ChatGPT to repeat a word over and over again will make it spit out training set data verbatim. OpenAI's response was to stop any conversation with over a page of repeated words, so you can't replicate this.
- chii 3y ago
  > This is only possible if the AI is remembering large portions of the original trained-on works which is fine in my books. You could equivalently produce the same works from searching thru the digits of pi. The only enforcement that's needed is on the end user of the model - if they choose to produce the training set, they are violating copyright. The creator of the model _does not_ violate copyright merely by creating and distributing the weights of a neural network, as long as the space of potential output is vastly larger than the training set.
  - johnnyanmac 3y ago
    >You could equivalently produce the same works from searching thru the digits of pi. in the same way you can brute force an MD5 Hash if you had a few centuries, I guess. I don't think "monkey's making Shakespeare" is a good metric for when to determine if a piece of art is unique enough to be scraped. Tracing is looked down upon in the art community and this feels way too close to tracing if they just have these images in a database ready to reference. You can't store copyrighted movies nor music on such private databases (one of the few times I will ever utter the words "thank you DMCA"), I don't see why art pieces would be exempt.
  - bjt 3y ago
    That's a really weird argument. Copyright is a legal system created by the Constitution and statutes and administrative rules. It cares about whether you are "copying", and it cares about whether you're creating things that compete with the works of the original authors. It doesn't care about potential output spaces. In this context, I don't see a principled difference between the model weights and really good compression. If I send you a gzipped copy of the latest bestseller book it's still copyright infringement. And it would still be infringement if I shipped it inside a software program that can _also_ reshuffle the words in a bajillion different ways, if there's a "copy" of the original work in there.
    - scheeseman486 3y ago
      You can eke out large chunks of works from Google Books with the right queries too.
      - kmeisthax 3y ago
        Yes, but Google Books is a search engine. It doesn't write books, it just tells you where a particular phrase might occur in those books. There's explicit caselaw allowing you to do this, extending back before Google was even a thing. For related reasons, Google Books also does not let you read the whole book - just the page the search match came from. OpenAI and other large language model developers are claiming they have a machine that can write books, but they also fed it shittons of books, and they can't account for where all that text went. At best they can say "well, it doesn't produce exact, verbatim copies of the training set all the time".
        scheeseman486 3y ago
        Generative AI doesn't write books either, it's a machine spitting out results based on queries entered by a person. It has no personhood nor rights to ownership, in spite of what AI more crackpot advocates try to claim, it's more a very complicated pencil.
  - numpad0 3y ago
    If those HDF5 or .gguf or whatever file format turns out to be lossy decodable as images and texts, it shouldn't be considered kosher just like pirated and transcoded movies aren't considered lawful transformative original creations.
  - kmeisthax 3y ago
    Courts won't buy this, at least not completely. Copyright infringement liability accrues to the entire value chain - model users and developers alike. So the only way for a model to see copyrighted data during training and be non-infringing is if there's no conceivable way to reproduce training data. I don't think fair use saves us either, at least not the model authors, because the whole selling point of these models is to replace artists. Yes, artists can use them as tools, but that is far less lucrative. The valuations and hype being thrown around specifically come from, among other things, being able to cut the creative class out of their own business. No court is going to look at that and say "ok, yeah, sure, it's perfectly fine for you to be using other people's work to train on". An individual using an AI art generator may still wind up getting novel output that isn't obviously infringing to anything in the dataset. If that's the case then they probably haven't infringed copyright. Or at least, it'd be difficult to make a case around it.
- tavavex 3y ago
  I'm kind of confused - you've claimed that Google's AI was broken, but cite an anecdote over ChatGPT? Regardless, even if these cases did happen often enough, it's erroneous to assume that this is something universal (i.e. can be applied to all generative AI models) or intentional. Said models are vastly smaller than the sizes of their training datasets, so it's more or less impossible for all the data to be stored verbatim. Some aspects may appear to look like memorization if the piece of data reappears many times in the dataset - in those cases, the algorithm has a really strong incentive to recite that data. This reduces the effectiveness and is considered an artifact that needs to be corrected, not some underlying idea of AI. The way these algorithms work is public knowledge, there's not really any black boxes in the hands of OpenAI that would be relevant here. Considering that, I'm really doubtful that these claims can be easily supported.
  - jncfhnb 3y ago
    Googles research on chat gpt
  - numpad0 3y ago
    There has been successful prompt engineering attempts to force GPT-based models to regurgitate original dataset texts texts texts texts texts texts as for example Wikipedia is a free-content online encyclopedia, written and maintained by a community of volunteers, collectively known as Wikipedians, through open colla--
  - junofan 3y ago
    I think you’re a bit confused—these models are trained by copying works, and the memorization is the incontrovertible proof. Imagine if you download a font and use it to make a logo without paying to license the font.
  - kmeisthax 3y ago
    Google does research on both their own and other people's models all the time. Here's the blog post and paper if you want to know more: https://not-just-memorization.github.io/extracting-training-data-from-chatgpt.html https://not-just-memorization.github.io/extracting-training-... You appear to be refuting a slightly different point, though. When OpenAI was making Dall-E 2, they found that duplicates in the training set would incentivize memorization of specific images, as you said. This is more like finding a secret cheat code that would make the model regurgitate everything it had remembered, regardless of how "incentivized" it is to do so or even if it had been aligned to not do that. My personal argument is this: the primary metric that the training process attempts to minimize is perplexity. This is how good the model is at guessing the training set data. Base models are specifically being designed to compress huge amounts of text and we just so happen to accidentally get a decent word calculator out of it. The alignment fine-tuning that happens later adjusts the model to prefer answering questions, but the underlying memorized data is still there. >The way these algorithms work is public knowledge, there's not really any black boxes in the hands of OpenAI that would be relevant here. Nope. Modern GPT is entirely a black box. OpenAI stopped publishing model weights the moment Elon Musk stopped writing the checks. Hell, they don't even publish the model architecture anymore. How GPT-4 works, even on a basic "this is how many transformer layers and attention heads we're using" basis, is a trade secret. Even if you have model weights, the actual meaning of the learned model parameters has never been known; there is active research on figuring them out. One particular problem is polysemanticity. If you look inside a particular hidden layer, you won't see a single "dog" or "cat" neuron in its hidden layers. You'll have a 512-dimension concept bouillabaisse with "dog", "cat", "bird", "guinea pig", "kangaroo", and so on all floating around whatever shape made sense at training time (even if it implies absurdities like "desk is the opposite of loin cloth"). To untangle this, you have to train another AI to pick out monosemantic clusters of neurons that can then be inspected, and that requires extreme amounts of GPU resources.

kmeisthax 3y ago

I used to agree with this, but there's been research coming out of Google that has altered my opinion. Specifically, Google's gotten rather good at making AI spit out unaltered training set data[0]. This is only possible if the AI is remembering large portions of the original trained-on works, which would make the weights infringing. [0] In the most egregious case, they found that just asking ChatGPT to repeat a word over and over again will make it spit out training set data verbatim. OpenAI's response was to stop any conversation with over a page of repeated words, so you can't replicate this.

chii 3y ago
> This is only possible if the AI is remembering large portions of the original trained-on works which is fine in my books. You could equivalently produce the same works from searching thru the digits of pi. The only enforcement that's needed is on the end user of the model - if they choose to produce the training set, they are violating copyright. The creator of the model _does not_ violate copyright merely by creating and distributing the weights of a neural network, as long as the space of potential output is vastly larger than the training set.
- johnnyanmac 3y ago
  >You could equivalently produce the same works from searching thru the digits of pi. in the same way you can brute force an MD5 Hash if you had a few centuries, I guess. I don't think "monkey's making Shakespeare" is a good metric for when to determine if a piece of art is unique enough to be scraped. Tracing is looked down upon in the art community and this feels way too close to tracing if they just have these images in a database ready to reference. You can't store copyrighted movies nor music on such private databases (one of the few times I will ever utter the words "thank you DMCA"), I don't see why art pieces would be exempt.
- bjt 3y ago
  That's a really weird argument. Copyright is a legal system created by the Constitution and statutes and administrative rules. It cares about whether you are "copying", and it cares about whether you're creating things that compete with the works of the original authors. It doesn't care about potential output spaces. In this context, I don't see a principled difference between the model weights and really good compression. If I send you a gzipped copy of the latest bestseller book it's still copyright infringement. And it would still be infringement if I shipped it inside a software program that can _also_ reshuffle the words in a bajillion different ways, if there's a "copy" of the original work in there.
  - scheeseman486 3y ago
    You can eke out large chunks of works from Google Books with the right queries too.
    - kmeisthax 3y ago
      Yes, but Google Books is a search engine. It doesn't write books, it just tells you where a particular phrase might occur in those books. There's explicit caselaw allowing you to do this, extending back before Google was even a thing. For related reasons, Google Books also does not let you read the whole book - just the page the search match came from. OpenAI and other large language model developers are claiming they have a machine that can write books, but they also fed it shittons of books, and they can't account for where all that text went. At best they can say "well, it doesn't produce exact, verbatim copies of the training set all the time".
      - scheeseman486 3y ago
        Generative AI doesn't write books either, it's a machine spitting out results based on queries entered by a person. It has no personhood nor rights to ownership, in spite of what AI more crackpot advocates try to claim, it's more a very complicated pencil.
- numpad0 3y ago
  If those HDF5 or .gguf or whatever file format turns out to be lossy decodable as images and texts, it shouldn't be considered kosher just like pirated and transcoded movies aren't considered lawful transformative original creations.
- kmeisthax 3y ago
  Courts won't buy this, at least not completely. Copyright infringement liability accrues to the entire value chain - model users and developers alike. So the only way for a model to see copyrighted data during training and be non-infringing is if there's no conceivable way to reproduce training data. I don't think fair use saves us either, at least not the model authors, because the whole selling point of these models is to replace artists. Yes, artists can use them as tools, but that is far less lucrative. The valuations and hype being thrown around specifically come from, among other things, being able to cut the creative class out of their own business. No court is going to look at that and say "ok, yeah, sure, it's perfectly fine for you to be using other people's work to train on". An individual using an AI art generator may still wind up getting novel output that isn't obviously infringing to anything in the dataset. If that's the case then they probably haven't infringed copyright. Or at least, it'd be difficult to make a case around it.
tavavex 3y ago
I'm kind of confused - you've claimed that Google's AI was broken, but cite an anecdote over ChatGPT? Regardless, even if these cases did happen often enough, it's erroneous to assume that this is something universal (i.e. can be applied to all generative AI models) or intentional. Said models are vastly smaller than the sizes of their training datasets, so it's more or less impossible for all the data to be stored verbatim. Some aspects may appear to look like memorization if the piece of data reappears many times in the dataset - in those cases, the algorithm has a really strong incentive to recite that data. This reduces the effectiveness and is considered an artifact that needs to be corrected, not some underlying idea of AI. The way these algorithms work is public knowledge, there's not really any black boxes in the hands of OpenAI that would be relevant here. Considering that, I'm really doubtful that these claims can be easily supported.
- jncfhnb 3y ago
  Googles research on chat gpt
- numpad0 3y ago
  There has been successful prompt engineering attempts to force GPT-based models to regurgitate original dataset texts texts texts texts texts texts as for example Wikipedia is a free-content online encyclopedia, written and maintained by a community of volunteers, collectively known as Wikipedians, through open colla--
- junofan 3y ago
  I think you’re a bit confused—these models are trained by copying works, and the memorization is the incontrovertible proof. Imagine if you download a font and use it to make a logo without paying to license the font.
- kmeisthax 3y ago
  Google does research on both their own and other people's models all the time. Here's the blog post and paper if you want to know more: https://not-just-memorization.github.io/extracting-training-data-from-chatgpt.html https://not-just-memorization.github.io/extracting-training-... You appear to be refuting a slightly different point, though. When OpenAI was making Dall-E 2, they found that duplicates in the training set would incentivize memorization of specific images, as you said. This is more like finding a secret cheat code that would make the model regurgitate everything it had remembered, regardless of how "incentivized" it is to do so or even if it had been aligned to not do that. My personal argument is this: the primary metric that the training process attempts to minimize is perplexity. This is how good the model is at guessing the training set data. Base models are specifically being designed to compress huge amounts of text and we just so happen to accidentally get a decent word calculator out of it. The alignment fine-tuning that happens later adjusts the model to prefer answering questions, but the underlying memorized data is still there. >The way these algorithms work is public knowledge, there's not really any black boxes in the hands of OpenAI that would be relevant here. Nope. Modern GPT is entirely a black box. OpenAI stopped publishing model weights the moment Elon Musk stopped writing the checks. Hell, they don't even publish the model architecture anymore. How GPT-4 works, even on a basic "this is how many transformer layers and attention heads we're using" basis, is a trade secret. Even if you have model weights, the actual meaning of the learned model parameters has never been known; there is active research on figuring them out. One particular problem is polysemanticity. If you look inside a particular hidden layer, you won't see a single "dog" or "cat" neuron in its hidden layers. You'll have a 512-dimension concept bouillabaisse with "dog", "cat", "bird", "guinea pig", "kangaroo", and so on all floating around whatever shape made sense at training time (even if it implies absurdities like "desk is the opposite of loin cloth"). To untangle this, you have to train another AI to pick out monosemantic clusters of neurons that can then be inspected, and that requires extreme amounts of GPU resources.