6 ms·

Why bother with the weights when you can just take the literal output of another model and consume as part of your training corpus? What kind of copyright clai

Why bother with the weights when you can just take the literal output of another model and consume as part of your training corpus? What kind of copyright claims can be made against the content generated by a model wielding the desired weights?

sva_ 4y ago
In fine-tuned models, for example, you may only train a real subset of the weights. Who is the copyright owner now and what is the justification? There might be other ways of transfer-learning where you can greatly benefit from the pretrained weights, some of which may have not yet been discovered.
- bob1029 4y ago
  > In fine-tuned models, for example, you may only train a real subset of the weights. I am not sure to what degree this applies throughout the various offerings. We are finding that "fine" tuning cuts pretty damn deep. For instance, with OpenAI you can start with base Curie/Davinci and build an effective chat bot that makes ChatGPT look incompetent in a very narrow vertical slice of some specialized domain.
sam0x17 4y ago
According to the US Copyright office's recent ruling, nothing generated by these models can be copyrighted (!), so in theory you should be fine!
- chatmasta 4y ago
  nit: that wasn't a "ruling" so much as it was "guidance" [0] [1] [0] https://www.federalregister.gov/documents/2023/03/16/2023-05321/copyright-registration-guidance-works-containing-material-generated-by-artificial-intelligence https://www.federalregister.gov/documents/2023/03/16/2023-05... [1] Discussed on HN: https://news.ycombinator.com/item?id=35191206 https://news.ycombinator.com/item?id=35191206
  - Zuiii 4y ago
    The position of the US copyright office is based in large part on prior case law (from your first link).
- avereveard 4y ago
  Yeah but gpt3-turbo has a "no train on our outputs" tos, so it's not just copyright.
  - computerex 4y ago
    Let's say company A uses OpenAI models to create an open dataset, like alpaca. If company B uses that dataset to fine tune a model, is company B liable to TOS violation? Afterall, they may not even be an OpenAI customer.
  - bob1029 4y ago
    Does it actually? The only thing I ever saw was "don't use our models to build competing models". Which is clearly ambiguous at best when dealing with generative natural text. This also implies to me that using their models to help with fine tunes of their other offerings would be ok.
- astrange 4y ago
  They're the first word on this, not the last, so it doesn't mean much. It also clearly leads to absurd results for text models, which can memorize their training data, or echo back your inputs. Or you could pass their output through a human to copyright it.
  - anticensor 4y ago
    Not all countries have the same view on this either. A few countries, including the UK, maintain the view that the AI's operator, owner or custodian hold the copyright instead.

sva_ 4y ago

In fine-tuned models, for example, you may only train a real subset of the weights. Who is the copyright owner now and what is the justification? There might be other ways of transfer-learning where you can greatly benefit from the pretrained weights, some of which may have not yet been discovered.

bob1029 4y ago
> In fine-tuned models, for example, you may only train a real subset of the weights. I am not sure to what degree this applies throughout the various offerings. We are finding that "fine" tuning cuts pretty damn deep. For instance, with OpenAI you can start with base Curie/Davinci and build an effective chat bot that makes ChatGPT look incompetent in a very narrow vertical slice of some specialized domain.

sam0x17 4y ago

According to the US Copyright office's recent ruling, nothing generated by these models can be copyrighted (!), so in theory you should be fine!

chatmasta 4y ago
nit: that wasn't a "ruling" so much as it was "guidance" [0] [1] [0] https://www.federalregister.gov/documents/2023/03/16/2023-05321/copyright-registration-guidance-works-containing-material-generated-by-artificial-intelligence https://www.federalregister.gov/documents/2023/03/16/2023-05... [1] Discussed on HN: https://news.ycombinator.com/item?id=35191206 https://news.ycombinator.com/item?id=35191206
- Zuiii 4y ago
  The position of the US copyright office is based in large part on prior case law (from your first link).
avereveard 4y ago
Yeah but gpt3-turbo has a "no train on our outputs" tos, so it's not just copyright.
- computerex 4y ago
  Let's say company A uses OpenAI models to create an open dataset, like alpaca. If company B uses that dataset to fine tune a model, is company B liable to TOS violation? Afterall, they may not even be an OpenAI customer.
- bob1029 4y ago
  Does it actually? The only thing I ever saw was "don't use our models to build competing models". Which is clearly ambiguous at best when dealing with generative natural text. This also implies to me that using their models to help with fine tunes of their other offerings would be ok.
astrange 4y ago
They're the first word on this, not the last, so it doesn't mean much. It also clearly leads to absurd results for text models, which can memorize their training data, or echo back your inputs. Or you could pass their output through a human to copyright it.
- anticensor 4y ago
  Not all countries have the same view on this either. A few countries, including the UK, maintain the view that the AI's operator, owner or custodian hold the copyright instead.