3 ms·

What's to stop somebody using the output at scale on local hardware to distill their own model and then making that available open weights?

by chr15m 5d ago

gspr 5d ago
This is the fundamental problem with where the AI race is heading, IMHO. Broadly, there are two possible legal interpretations (to my layman's mind): * A model is derived work of its training data. This seems sane to me. Open, but copyrighted, works (like FOSS) remain protected from abuse. There's some legal moat around AI models. But on the other hand it seems unlikely that there's enough liberally licensed (or public domain) training data to go around. The little guy's status quo remains, the frontier labs' work slows down massively. * A model is not derived work of its training data. This seems to me insane, but a lot of the world seems to hold this view (including the frontier labs). Stuff like FOSS or indie art is under huge threat of copyrightwashing. But on the other hand, there's also zero legal moat around the models. The little guy is eviscerated, but so are the frontier labs. Neither interpretation seems, to me, to be capable of sustaining the last couple of years' developments. But what do I know.
- skykooler 5d ago
  If a model is derived work of its training data, surely all the existing frontier models that have been training on copyrighted work have a big legal issue, and therefore so does anything produced with them?
  - gspr 5d ago
    Yes. Of course. They'd have to be rebuilt with acceptably licensed training data. And since there might not be a enough of it, the model owners are screwed. My point is that they're also screwed in the opposite scenario, because they rely on the same legal protection (against deriving works) as the works they trained on. That's why I don't understand how any of this can be sustained.
- dale_glass 5d ago
  > A model is not derived work of its training data. This seems to me insane, but a lot of the world seems to hold this view Why insane? Models don't take the content as-is, they take measurements. I don't owe you royalties just because I used your photo to get the proportions and coloring of a duck right. Go watch artist streams, you'll often see people to go Google Images for references. I've never seen that result in credit or payments. The alternative is that we hand out lots of money to a few large companies specializing in content archives, and there's really no benefit to anyone else anyway. On the long term I would expect a few fat cats to get fatter, the small guy to get nothing, and AI still work but get there slowly. I don't see the point or the benefit.
  - gspr 5d ago
    > Models don't take the content as-is, they take measurements. At some point, enough measurements constitute a copy. If I redistribute the average value of all the pixels in your photo, I'm obviously not in violation of your copyright. If I measure and redistribute 90% of its DCT coefficients (i.e. make a slightly compressed JPEG), I am. The interesting stuff happens between those extremes. We cannot just take as a given that all LLMs always are on the safe side. It is not at all obvious.
  - jml78 5d ago
    Yeah but here is the thing. Say you train a Lora for a model. You then merge those weights into the open weight model. Now prompt it for an original image, it will pretty much be able to reproduce that exact image. You can say it is just measurements but at some point, it can just reproduce with high enough accuracy to just be seen as a copy Given that corporations buy up any valuable IP, I personally think the answer is to abolish copyright because right now it is really only protecting the rich and corporations . Individuals have the illusion of protection but if Disney steals your shit, good luck with the pain and suffering you experience trying to win a court case against them
    - boplicity 5d ago
      Abolishing copyright would make it much harder for smaller players to protect their interests. Their original work would be gobbled up by the big players, and made easily available under the umbrella of a large corporations pre-existing market share. Not a fun situation at all to be in. Especially for the little guys.

What's to stop somebody using the output at scale on local hardware to distill their own model and then making that available open weights?

by chr15m 5d ago

What's to stop somebody using the output at scale on local hardware to distill their own model and then making that available open weights?

gspr 5d ago

This is the fundamental problem with where the AI race is heading, IMHO. Broadly, there are two possible legal interpretations (to my layman's mind): * A model is derived work of its training data. This seems sane to me. Open, but copyrighted, works (like FOSS) remain protected from abuse. There's some legal moat around AI models. But on the other hand it seems unlikely that there's enough liberally licensed (or public domain) training data to go around. The little guy's status quo remains, the frontier labs' work slows down massively. * A model is not derived work of its training data. This seems to me insane, but a lot of the world seems to hold this view (including the frontier labs). Stuff like FOSS or indie art is under huge threat of copyrightwashing. But on the other hand, there's also zero legal moat around the models. The little guy is eviscerated, but so are the frontier labs. Neither interpretation seems, to me, to be capable of sustaining the last couple of years' developments. But what do I know.

skykooler 5d ago
If a model is derived work of its training data, surely all the existing frontier models that have been training on copyrighted work have a big legal issue, and therefore so does anything produced with them?
- gspr 5d ago
  Yes. Of course. They'd have to be rebuilt with acceptably licensed training data. And since there might not be a enough of it, the model owners are screwed. My point is that they're also screwed in the opposite scenario, because they rely on the same legal protection (against deriving works) as the works they trained on. That's why I don't understand how any of this can be sustained.
dale_glass 5d ago
> A model is not derived work of its training data. This seems to me insane, but a lot of the world seems to hold this view Why insane? Models don't take the content as-is, they take measurements. I don't owe you royalties just because I used your photo to get the proportions and coloring of a duck right. Go watch artist streams, you'll often see people to go Google Images for references. I've never seen that result in credit or payments. The alternative is that we hand out lots of money to a few large companies specializing in content archives, and there's really no benefit to anyone else anyway. On the long term I would expect a few fat cats to get fatter, the small guy to get nothing, and AI still work but get there slowly. I don't see the point or the benefit.
- gspr 5d ago
  > Models don't take the content as-is, they take measurements. At some point, enough measurements constitute a copy. If I redistribute the average value of all the pixels in your photo, I'm obviously not in violation of your copyright. If I measure and redistribute 90% of its DCT coefficients (i.e. make a slightly compressed JPEG), I am. The interesting stuff happens between those extremes. We cannot just take as a given that all LLMs always are on the safe side. It is not at all obvious.
- jml78 5d ago
  Yeah but here is the thing. Say you train a Lora for a model. You then merge those weights into the open weight model. Now prompt it for an original image, it will pretty much be able to reproduce that exact image. You can say it is just measurements but at some point, it can just reproduce with high enough accuracy to just be seen as a copy Given that corporations buy up any valuable IP, I personally think the answer is to abolish copyright because right now it is really only protecting the rich and corporations . Individuals have the illusion of protection but if Disney steals your shit, good luck with the pain and suffering you experience trying to win a court case against them
  - boplicity 5d ago
    Abolishing copyright would make it much harder for smaller players to protect their interests. Their original work would be gobbled up by the big players, and made easily available under the umbrella of a large corporations pre-existing market share. Not a fun situation at all to be in. Especially for the little guys.