4 ms·

Proprietary model weights are IP, their outputs are not IP and I can't imagine a court decision that would rule otherwise because that would set an extremely fa

Proprietary model weights are IP, their outputs are not IP and I can't imagine a court decision that would rule otherwise because that would set an extremely far reaching and dangerous precedent, even for American businesses. I don't think distillation as 'stealing IP' has any legal legs. They can probably claim violation of ToS at best because the terms of use do prohibit use for training rival models.

EMIRELADERO 3mo ago
> Proprietary model weights are IP I don't think they are. Not copyright at least. There may be some "trade secret" stuff for them but they're not copyrightable.
- delecti 3mo ago
  Trade secrets are a type of IP.
  - masfuerte 3mo ago
    Yes, but if a company publishes its trade secrets on the web you are entitled to read them and use them.
    - munk-a 3mo ago
      Only if you're going to use them to train an LLM! Also, more seriously, there are plenty of pieces of information that are compelled to be published that don't lose their confidentiality. There actually is a general concept that confidentiality can be lost if you are negligent in its protection but it's a very nuanced thing.
  - californical 3mo ago
    Sure, just like it’s illegal to steal the formula of Coca Cola from their documents. But you can do an analysis of a bottle of Coke to figure out exactly what’s in it. You can even make your own and sell it, if you call it a different name.
- downrightmike 3mo ago
  And if an AI generated those wights, they are not copyrightable
  - greensoap 3mo ago
    Even if the weights are just made through traditional training and no AI is used to make the weights... is there a copyright right? If the weights are a numerical derivation that aren't made by a human is there even human expression? Absent human expression, there is no copyright right?
    - kube-system 3mo ago
      The output being created by a machine is not automatically disqualifying. e.g. PCM audio data is all created by a machine, but that doesn't prevent CDs from being copyrighted... because the input to it was highly human-created. Ultimately it all hinges on the specifics of the training and how much human involvement was involved throughout the creation process. I suspect arguing this successfully in favor of upholding a copyright would be an uphill battle for many situations, but ultimately we'll need more court cases to know for sure.
  - kube-system 3mo ago
    That could effectively be the case here but your statement in isolation is an oversimplification of recent legal guidance on this matter -- AI generated things are copyrightable if there is sufficient human authorship involved in the generation process. The copyright office's recent statement on this matter were sensationalized when they really said nothing groundbreaking at all -- this was always the standard applied when any tools are used in creating a work.
- Zigurd 3mo ago
  Trade secrets are easy to claim but hard to turn into a tort without having someone steal them. I'm sure some lawyer can come up with a filing that tries to make distillation an act of theft, but there would be many weak links in that chain. Contrast this with Apple's case where it looks like they've got evidence of people walking out of the building with various physical artifacts on their way to an OpenAI interview.
sillysaurusx 3mo ago
> Proprietary model weights are IP Not copyrightable IP, or at least it hasn't been challenged yet. I have experience with this: I made llama-dl, a way to download the original llama model. Meta issued a DMCA, I appealed to the HN community for funds, someone funded, and our lawyer successfully counterclaimed. Never heard from Meta again. A lack of response to the counterclaim doesn't mean the issue is settled. But now there's legal precedent for people pushing back against companies that claim model weights are secret IP and therefore DMCA-able.
- deaton 3mo ago
  I do wonder if their "fair use" argument preempts them from actually being able to claim copyright over the model weights, since it can be argued that the vast majority of what the model is is itself a remix of everything else. Maybe their RLHF, system prompts, architecture and training techniques can be considered copyrightable, but maybe most of the data can't.
- anon373839 3mo ago
  Closed model weights probably do qualify as trade secrets, though. But I agree 100% that anything extractable from the API outputs is not secret and does not qualify. And they certainly aren’t copyrighted or patented.
- kube-system 3mo ago
  Congrats on your efforts, but by definition that does not count as legal precedent. At a bare minimum, a legal precedent is some sort of written holding by a court with authority. Even an out-of-court settlement doesn't count.
  - sillysaurusx 3mo ago
    It was referenced by other legal cases. Why would they mention it if it didn’t help their case?
    - kube-system 3mo ago
stusmall 3mo ago
I can't wrap my head around the idea that distillation is IP theft but mass training on books, music and art without consent is fair use. The two stances are incompatible. If it is transformative use of a book, it is transformative use of AI output.
- munk-a 3mo ago
  I really hope that when a sane administration returns to power in the US we'll actually get a reckoning over how irrational it was to not restrict training data.
  - hirako2000 3mo ago
    It isn't irrational. The idea is that selling, or giving away art grants the consumer to sell art of his own, not a clear copy of. So the court rationally found Anthropic guilty for acquiring copies without paying for them. But didn't charge the training, deemed fair use. If it isn't fair use, then we may need to sue all teachers for spitting out knowledge they ultimately acquired from someone's work. Copyright, some would say is irrational. Humans learn, that is copying, distilling in fact. Copyright though is pragmatic: it draws a line, art will be reproduced, let's just enforce that they can't be shameless (near) identical copies. Mix it up, derive the original enough so that you aren't competing with the original author. The only argument to ban training on copyright data is that it unfairly compete with original authors. Which stance do you take? Neither would be irrational.
    - californical 3mo ago
      I agree that the fair use argument on training data is a genie that’s not getting put back in a bottle. But: > If it isn't fair use, then we may need to sue all teachers for spitting out knowledge they ultimately acquired from someone's work. Is such a lazy argument and always was. A human doing something is necessarily different than a machine doing it. We can be ok with a human doing a thing and simultaneously not ok with a machine doing it.
munk-a 3mo ago
> their outputs are not IP and I can't imagine a court decision that would rule otherwise because that would set an extremely far reaching and dangerous precedent Assume that their model output is considered IP and that it's ruled illegal to train on that IP. I will offer to sell every content producer on earth an identity LLM that takes their content and outputs precisely identical content that they can then post. Good luck ever getting any training data for free ever again.
temporalparts 3mo ago
Setting aside the strict definition of IP, this does satisfy the philosophical motivation of IP, which is to protect people making significant capital investments. This is not too different from drug discovery where it's extremely difficult to come up with the molecule, but relatively easy to copy it. Similarly, it's really hard to create frontier models from scratch but much easier to distill them.
- cdata 3mo ago
  I'm all for framing the spirit this way, so long as we recognize that it was rather difficult to come up with the sum total of human creative output (noting also that approximately zero of the cost to produce it has been born by frontier labs so far).
rocqua 3mo ago
Your looking at this from a legal perspective. Consider a diplomatic one instead. Model training is expensive. Model distillation is cheaper. If you train models, and that lets your adversary distill them for much cheaper, that gives your adversary an advantage. Giving your adversary an advantage is bad, so you address it. It doesn’t matter whether the advantage was gained legally, not even if it was fair. The advantage in and of itself is bad, and sufficient reason to act. You couch it in less nakedly power hungry terms, but this is the reasoning.
- aembleton 3mo ago
  If they are an adversary, then you could inject broken information for them to distil with. Then have an alternative, private model for government and military purposes that doesn't have the broken information.

EMIRELADERO 3mo ago

> Proprietary model weights are IP I don't think they are. Not copyright at least. There may be some "trade secret" stuff for them but they're not copyrightable.

delecti 3mo ago
Trade secrets are a type of IP.
- masfuerte 3mo ago
  Yes, but if a company publishes its trade secrets on the web you are entitled to read them and use them.
  - munk-a 3mo ago
    Only if you're going to use them to train an LLM! Also, more seriously, there are plenty of pieces of information that are compelled to be published that don't lose their confidentiality. There actually is a general concept that confidentiality can be lost if you are negligent in its protection but it's a very nuanced thing.
- californical 3mo ago
  Sure, just like it’s illegal to steal the formula of Coca Cola from their documents. But you can do an analysis of a bottle of Coke to figure out exactly what’s in it. You can even make your own and sell it, if you call it a different name.
downrightmike 3mo ago
And if an AI generated those wights, they are not copyrightable
- greensoap 3mo ago
  Even if the weights are just made through traditional training and no AI is used to make the weights... is there a copyright right? If the weights are a numerical derivation that aren't made by a human is there even human expression? Absent human expression, there is no copyright right?
  - kube-system 3mo ago
    The output being created by a machine is not automatically disqualifying. e.g. PCM audio data is all created by a machine, but that doesn't prevent CDs from being copyrighted... because the input to it was highly human-created. Ultimately it all hinges on the specifics of the training and how much human involvement was involved throughout the creation process. I suspect arguing this successfully in favor of upholding a copyright would be an uphill battle for many situations, but ultimately we'll need more court cases to know for sure.
- kube-system 3mo ago
  That could effectively be the case here but your statement in isolation is an oversimplification of recent legal guidance on this matter -- AI generated things are copyrightable if there is sufficient human authorship involved in the generation process. The copyright office's recent statement on this matter were sensationalized when they really said nothing groundbreaking at all -- this was always the standard applied when any tools are used in creating a work.
Zigurd 3mo ago
Trade secrets are easy to claim but hard to turn into a tort without having someone steal them. I'm sure some lawyer can come up with a filing that tries to make distillation an act of theft, but there would be many weak links in that chain. Contrast this with Apple's case where it looks like they've got evidence of people walking out of the building with various physical artifacts on their way to an OpenAI interview.

sillysaurusx 3mo ago

> Proprietary model weights are IP Not copyrightable IP, or at least it hasn't been challenged yet. I have experience with this: I made llama-dl, a way to download the original llama model. Meta issued a DMCA, I appealed to the HN community for funds, someone funded, and our lawyer successfully counterclaimed. Never heard from Meta again. A lack of response to the counterclaim doesn't mean the issue is settled. But now there's legal precedent for people pushing back against companies that claim model weights are secret IP and therefore DMCA-able.

deaton 3mo ago
I do wonder if their "fair use" argument preempts them from actually being able to claim copyright over the model weights, since it can be argued that the vast majority of what the model is is itself a remix of everything else. Maybe their RLHF, system prompts, architecture and training techniques can be considered copyrightable, but maybe most of the data can't.
anon373839 3mo ago
Closed model weights probably do qualify as trade secrets, though. But I agree 100% that anything extractable from the API outputs is not secret and does not qualify. And they certainly aren’t copyrighted or patented.
kube-system 3mo ago
Congrats on your efforts, but by definition that does not count as legal precedent. At a bare minimum, a legal precedent is some sort of written holding by a court with authority. Even an out-of-court settlement doesn't count.
- sillysaurusx 3mo ago
  It was referenced by other legal cases. Why would they mention it if it didn’t help their case?
  - kube-system 3mo ago

stusmall 3mo ago

I can't wrap my head around the idea that distillation is IP theft but mass training on books, music and art without consent is fair use. The two stances are incompatible. If it is transformative use of a book, it is transformative use of AI output.

munk-a 3mo ago
I really hope that when a sane administration returns to power in the US we'll actually get a reckoning over how irrational it was to not restrict training data.
- hirako2000 3mo ago
  It isn't irrational. The idea is that selling, or giving away art grants the consumer to sell art of his own, not a clear copy of. So the court rationally found Anthropic guilty for acquiring copies without paying for them. But didn't charge the training, deemed fair use. If it isn't fair use, then we may need to sue all teachers for spitting out knowledge they ultimately acquired from someone's work. Copyright, some would say is irrational. Humans learn, that is copying, distilling in fact. Copyright though is pragmatic: it draws a line, art will be reproduced, let's just enforce that they can't be shameless (near) identical copies. Mix it up, derive the original enough so that you aren't competing with the original author. The only argument to ban training on copyright data is that it unfairly compete with original authors. Which stance do you take? Neither would be irrational.
  - californical 3mo ago
    I agree that the fair use argument on training data is a genie that’s not getting put back in a bottle. But: > If it isn't fair use, then we may need to sue all teachers for spitting out knowledge they ultimately acquired from someone's work. Is such a lazy argument and always was. A human doing something is necessarily different than a machine doing it. We can be ok with a human doing a thing and simultaneously not ok with a machine doing it.

munk-a 3mo ago

> their outputs are not IP and I can't imagine a court decision that would rule otherwise because that would set an extremely far reaching and dangerous precedent Assume that their model output is considered IP and that it's ruled illegal to train on that IP. I will offer to sell every content producer on earth an identity LLM that takes their content and outputs precisely identical content that they can then post. Good luck ever getting any training data for free ever again.

temporalparts 3mo ago

Setting aside the strict definition of IP, this does satisfy the philosophical motivation of IP, which is to protect people making significant capital investments. This is not too different from drug discovery where it's extremely difficult to come up with the molecule, but relatively easy to copy it. Similarly, it's really hard to create frontier models from scratch but much easier to distill them.

cdata 3mo ago
I'm all for framing the spirit this way, so long as we recognize that it was rather difficult to come up with the sum total of human creative output (noting also that approximately zero of the cost to produce it has been born by frontier labs so far).

rocqua 3mo ago

Your looking at this from a legal perspective. Consider a diplomatic one instead. Model training is expensive. Model distillation is cheaper. If you train models, and that lets your adversary distill them for much cheaper, that gives your adversary an advantage. Giving your adversary an advantage is bad, so you address it. It doesn’t matter whether the advantage was gained legally, not even if it was fair. The advantage in and of itself is bad, and sufficient reason to act. You couch it in less nakedly power hungry terms, but this is the reasoning.

aembleton 3mo ago
If they are an adversary, then you could inject broken information for them to distil with. Then have an alternative, private model for government and military purposes that doesn't have the broken information.