5 ms·

All the model's weights are just a compressed representation of data it was trained on. Keep in mind it's open data, created by millions of people for free and

All the model's weights are just a compressed representation of data it was trained on. Keep in mind it's open data, created by millions of people for free and placed either on Stackexchange or Reddit or Facebook or on Wikipedia. Thus, it is a nonsense to speak about any kind of copyright wrt neural networks.

lbotos 4y ago
Can’t wait until you learn that you can copyright a database. As in a collection data and it’s relation fixed in a tangible medium. So yes, by current definition model weights are absolutely eligible for copyright.
- BaseballPhysics 4y ago
  The key is that it's a fixed representation. But if I pull your weights and create my own representation, it wouldn't surprise me if that was not considered a derivative work, in the same way I can publish someone else's recipe so long as I rewrite it. In the end we'll see, but I wouldn't be at all surprised if courts ruled that the data itself was not protected.
  - lbotos 4y ago
    Recipes are not databases. You can repost a recipe straight up. The recipe itself is not copyrightable. The blog post/cookbook/etc which has a recipe embedded could be. (Copyrighted works can contain uncopyrightable elements)
    - BaseballPhysics 4y ago
      That's not at all true. If I take a picture of a recipe from a cookbook and post it, that is absolutely a copyright violation, as the specific presentation is a copyrighted work. This is because you can't copyright facts or procedures, but you can copyright specific representations/presentations of those facts or procedures. If I take that recipes, type it out, and post my own version? Yeah, that's totally fine. But I can't verbatim copy yours.
      - deleted 4y ago
        [deleted]
      - lbotos 4y ago
        You didn’t “copy the recipe” You copied the fixed representation of that recipe and reproduced it. Yes, we agree. That is copyright infringement. You do not have that right. Edit: I see my use of “repost” was the hinging confusion. Yes, that isn’t right. You can “reproduce the recipe”. When I said repost I meant copy and paste the text.
        BaseballPhysics 4y ago
        Be careful, even copying and pasting the text can get you in trouble depending on what's in that text (I suspect this is one of the reasons modern recipe websites include so much damn narrative crap, as it makes it easier to assert copyright ownership over the work), but I grant I'm kinda nitpicking at this point, I agree, I think we're on the same page. :)
      - Hizonner 4y ago
        > If I take that recipes, type it out, and post my own version? You probably need to make sure that you change any elements that aren't either common standards of how recipes are written ("preheat oven to 350 degrees"), or absolutely dictated by the requirement to convey the procedure. Which for a recipe probably just amounts to a paraphrase at most. But I don't think you can get away with copying every recipe absolutely verbatim. The real problem, of course, is that people with money, and adherents of certain ideologies presently popular among people with both money and power, are pressing for the widest possible definition of what's creative in every possible area, because they fundamentally think that the best policy is that absolutely everything should be ownable somehow. I suspect that the actual practical result will be that if, as should happen and very well may happen in some countries, courts decide that copyright law doesn't cover ML models, legislators will just extend copyright law until it does.
- notahacker 4y ago
  The trouble with anyone trying to copyright their model weights is that it's likely to make legal arguments their model isn't a derivative work of copyrighted inputs harder to sustain. Successfully copyrighting model weights (like databases or maps or other non-creative representations of factual inputted data) would be a very favourable outcome for creators of NNs who exclusively use licensed or public domain content, but not so much for a lot of the existing state of the art ML models...
  - lbotos 4y ago
    Sure. And there are companies now that are training models on licensed data and those models are def 100% copyrightable without question (it’s a database)
- jrochkind1 4y ago
  I think whether you can "copyright a database" has a somewhat more complicated answer than you suggest, especially in the USA. Was the "selection and arrangements" of the weights made by a human using at least some minimal level of creativity? I'm not really sure, it's an odd question. I'd lean toward no. https://sco.library.emory.edu/research-data-management/publishing/copyright-data.html https://sco.library.emory.edu/research-data-management/publi...
version_five 4y ago
All the model's weights are just a compressed representation of data it was trained on That's not remotely accurate. I'd argue it's not even understood what the weights are or how they relate to the training data, and what is understood is not at all trivial.
- BaseballPhysics 4y ago
  That's... just not true. We know these models capture aspects of the training dataset, otherwise training wouldn't be a thing. That's the entire point. We also know these models create a very compact representation of the data they derive. So it is perfectly reasonable to think of these things as a compressed representation. Is it lossy? Yes, of course. But so what?
  - tomrod 4y ago
    A valid model is f(all inputs imaginable)=2. No weights involved. You can apply as much training as you like, but its still just the value "2" as an output. There is no representation of the data. ML models are a mix of two things: 1. Patterns defined by the individuals performing the model training. Commonly this is referred to as model architecture, but ultimately its the cobbling together of various items to create a way to get a signal from data in a certain way. (As a side note, thanks Statisticians and Computer Scientists for leading the way here!) 2. Model weights to minimize the uncertainty in those patterns. How that uncertainty is measured is part of (1).
  - pixl97 4y ago
    >Is it lossy? Yes, of course. But so what? "But so what" is missing the entire point of intelligence. You as an intelligent creature only work because of this lossiness. Reality presents far too much information to process and instead brains, especially in higher lifeforms have evolved to filter out as much useless information as possible in the most energy efficient way, while not losing important context.
  - Riverheart 4y ago
    Right there's over/under fitting. Train SD on the same data too many times it over produces it in results. Just because they tailor it to get more nebulous results doesn't mean the data doesn't exist in some form.
michaelmrose 4y ago
Weights aren't a compressed representation of the data it was trained on because 99.99% of it can't be recovered verbatim. Also the fact that a piece of data is delivered to you free of cost doesn't mean they surrendered their copyright to you. This basically settles nothing.
- BaseballPhysics 4y ago
  > Weights aren't a compressed representation of the data it was trained on because 99.99% of it can't be recovered verbatim. Cool. So a low quality JPEG isn't a compressed representation?
  - zirgs 4y ago
    Good luck suing me if I resized your photo to 1x1 resolution and posted it on my website. That's how little info is extracted from each image on average.
    - BaseballPhysics 4y ago
      Cool, can I also make up statistics? How about we use actual examples: Getty is suing Stability because the model has captured so much information about its training set that it reproduces Getty watermarks in its results: https://arstechnica.com/tech-policy/2023/02/getty-sues-stability-ai-for-copying-12m-photos-and-imitating-famous-watermark/ https://arstechnica.com/tech-policy/2023/02/getty-sues-stabi... Github's Co-pilot can be coaxed into reproducing entire chunks of copyrighted code: https://twitter.com/docsparse/status/1581461734665367554 https://twitter.com/docsparse/status/1581461734665367554 But sure, go ahead, keep trying to convince me these models (particularly once they get large enough) aren't compressed representations of information represented in the underlying training data.
  - michaelmrose 4y ago
    The law isn't settled on this but whatever the final answer is it will clearly not be resolved by relying on poor analogy. A model is NOTHING like a jpeg. It's also little like a human being learning. We are going to need to be adults and pass actual legislation speaking to this.

lbotos 4y ago

Can’t wait until you learn that you can copyright a database. As in a collection data and it’s relation fixed in a tangible medium. So yes, by current definition model weights are absolutely eligible for copyright.

BaseballPhysics 4y ago
The key is that it's a fixed representation. But if I pull your weights and create my own representation, it wouldn't surprise me if that was not considered a derivative work, in the same way I can publish someone else's recipe so long as I rewrite it. In the end we'll see, but I wouldn't be at all surprised if courts ruled that the data itself was not protected.
- lbotos 4y ago
  Recipes are not databases. You can repost a recipe straight up. The recipe itself is not copyrightable. The blog post/cookbook/etc which has a recipe embedded could be. (Copyrighted works can contain uncopyrightable elements)
  - BaseballPhysics 4y ago
    That's not at all true. If I take a picture of a recipe from a cookbook and post it, that is absolutely a copyright violation, as the specific presentation is a copyrighted work. This is because you can't copyright facts or procedures, but you can copyright specific representations/presentations of those facts or procedures. If I take that recipes, type it out, and post my own version? Yeah, that's totally fine. But I can't verbatim copy yours.
    - deleted 4y ago
      [deleted]
    - lbotos 4y ago
      You didn’t “copy the recipe” You copied the fixed representation of that recipe and reproduced it. Yes, we agree. That is copyright infringement. You do not have that right. Edit: I see my use of “repost” was the hinging confusion. Yes, that isn’t right. You can “reproduce the recipe”. When I said repost I meant copy and paste the text.
      - BaseballPhysics 4y ago
        Be careful, even copying and pasting the text can get you in trouble depending on what's in that text (I suspect this is one of the reasons modern recipe websites include so much damn narrative crap, as it makes it easier to assert copyright ownership over the work), but I grant I'm kinda nitpicking at this point, I agree, I think we're on the same page. :)
    - Hizonner 4y ago
      > If I take that recipes, type it out, and post my own version? You probably need to make sure that you change any elements that aren't either common standards of how recipes are written ("preheat oven to 350 degrees"), or absolutely dictated by the requirement to convey the procedure. Which for a recipe probably just amounts to a paraphrase at most. But I don't think you can get away with copying every recipe absolutely verbatim. The real problem, of course, is that people with money, and adherents of certain ideologies presently popular among people with both money and power, are pressing for the widest possible definition of what's creative in every possible area, because they fundamentally think that the best policy is that absolutely everything should be ownable somehow. I suspect that the actual practical result will be that if, as should happen and very well may happen in some countries, courts decide that copyright law doesn't cover ML models, legislators will just extend copyright law until it does.
notahacker 4y ago
The trouble with anyone trying to copyright their model weights is that it's likely to make legal arguments their model isn't a derivative work of copyrighted inputs harder to sustain. Successfully copyrighting model weights (like databases or maps or other non-creative representations of factual inputted data) would be a very favourable outcome for creators of NNs who exclusively use licensed or public domain content, but not so much for a lot of the existing state of the art ML models...
- lbotos 4y ago
  Sure. And there are companies now that are training models on licensed data and those models are def 100% copyrightable without question (it’s a database)
jrochkind1 4y ago
I think whether you can "copyright a database" has a somewhat more complicated answer than you suggest, especially in the USA. Was the "selection and arrangements" of the weights made by a human using at least some minimal level of creativity? I'm not really sure, it's an odd question. I'd lean toward no. https://sco.library.emory.edu/research-data-management/publishing/copyright-data.html https://sco.library.emory.edu/research-data-management/publi...

version_five 4y ago

All the model's weights are just a compressed representation of data it was trained on That's not remotely accurate. I'd argue it's not even understood what the weights are or how they relate to the training data, and what is understood is not at all trivial.

BaseballPhysics 4y ago
That's... just not true. We know these models capture aspects of the training dataset, otherwise training wouldn't be a thing. That's the entire point. We also know these models create a very compact representation of the data they derive. So it is perfectly reasonable to think of these things as a compressed representation. Is it lossy? Yes, of course. But so what?
- tomrod 4y ago
  A valid model is f(all inputs imaginable)=2. No weights involved. You can apply as much training as you like, but its still just the value "2" as an output. There is no representation of the data. ML models are a mix of two things: 1. Patterns defined by the individuals performing the model training. Commonly this is referred to as model architecture, but ultimately its the cobbling together of various items to create a way to get a signal from data in a certain way. (As a side note, thanks Statisticians and Computer Scientists for leading the way here!) 2. Model weights to minimize the uncertainty in those patterns. How that uncertainty is measured is part of (1).
- pixl97 4y ago
  >Is it lossy? Yes, of course. But so what? "But so what" is missing the entire point of intelligence. You as an intelligent creature only work because of this lossiness. Reality presents far too much information to process and instead brains, especially in higher lifeforms have evolved to filter out as much useless information as possible in the most energy efficient way, while not losing important context.
- Riverheart 4y ago
  Right there's over/under fitting. Train SD on the same data too many times it over produces it in results. Just because they tailor it to get more nebulous results doesn't mean the data doesn't exist in some form.

michaelmrose 4y ago

Weights aren't a compressed representation of the data it was trained on because 99.99% of it can't be recovered verbatim. Also the fact that a piece of data is delivered to you free of cost doesn't mean they surrendered their copyright to you. This basically settles nothing.

BaseballPhysics 4y ago
> Weights aren't a compressed representation of the data it was trained on because 99.99% of it can't be recovered verbatim. Cool. So a low quality JPEG isn't a compressed representation?
- zirgs 4y ago
  Good luck suing me if I resized your photo to 1x1 resolution and posted it on my website. That's how little info is extracted from each image on average.
  - BaseballPhysics 4y ago
    Cool, can I also make up statistics? How about we use actual examples: Getty is suing Stability because the model has captured so much information about its training set that it reproduces Getty watermarks in its results: https://arstechnica.com/tech-policy/2023/02/getty-sues-stability-ai-for-copying-12m-photos-and-imitating-famous-watermark/ https://arstechnica.com/tech-policy/2023/02/getty-sues-stabi... Github's Co-pilot can be coaxed into reproducing entire chunks of copyrighted code: https://twitter.com/docsparse/status/1581461734665367554 https://twitter.com/docsparse/status/1581461734665367554 But sure, go ahead, keep trying to convince me these models (particularly once they get large enough) aren't compressed representations of information represented in the underlying training data.
- michaelmrose 4y ago
  The law isn't settled on this but whatever the final answer is it will clearly not be resolved by relying on poor analogy. A model is NOTHING like a jpeg. It's also little like a human being learning. We are going to need to be adults and pass actual legislation speaking to this.