3 ms·

But they won’t give us the model… so it’s ultimately meaningless because they’ll just sell out

itake 4y ago
My understanding is they legally can't. It was trained used OpenAI, which doesn't allow using their output to train new models. Someone would need to find another data source to fine tune llama.
- sdrg822 4y ago
  It’s only a matter of time
  - hackernewds 4y ago
    waiting for the stable diffusion version of gpt still
- dwallin 4y ago
  You don't need to find a new data source, you just need to find an unencumbered third party. You can use the that data publicly provided in the git repo as long as you haven't signed an agreement with OpenAI yourself.
- Zetice 4y ago
  What will OpenAI do, sue? Okay but now it's out there.
  - nwoli 4y ago
    Also how could that even be under protection. As if they haven’t been scraping copyrighted materials and sites with end user agreements to train the model in the first place
  - TaylorAlexander 4y ago
    I believe the work was done by Stanford, so OpenAI could revoke Stanford's access to their API. That would inhibit Stanford's ability to do new research with this system.
  - jerpint 4y ago
    OpenAI I knows they’re just a few cycles behind open source models - probably why they stuck a deal early with Microsoft.
- hawski 4y ago
  Why they can train their model on copyrighted data and claiming fair use as they do not outright copy while disallowing training other models on their output? I understand revoking access though.
  - alwayslikethis 4y ago
    They realistically can not, I believe. If Microsoft is right that this is under fair use, then you are only limited by OpenAI ToS, which also says that the copyright of generated content belongs to the user. Do the following: Person A uses GPT3 to generate training data, and publish it on his blog, without representing it as human generated. Person A does not give permission for it to be used by Alpaca team. Alpaca team comes along, scrape his blog, and uses it as training data, without permission from person A. Now this is fair use, so there is nothing person A can do to stop it, just like how Github scraped our code for Copilot without permission.
simonw 4y ago
If they don't release the model, recreating it doesn't look too hard. $100 worth of compute time to run the fine-tuning, and the training data they used is here: https://github.com/tatsu-lab/stanford_alpaca/blob/main/alpaca_data.json https://github.com/tatsu-lab/stanford_alpaca/blob/main/alpac... That would have the same licensing problems that they have though: that alpaca_data.json file was created using GPT3. But creating a "clean" training set of 52,000 examples doesn't feel impossible to me for the right group.
- dwallin 4y ago
  You're only bound by the terms of OpenAI's agreement if you agreed to the terms of use. If a third party obtained the data without signing an agreement with OpenAI (eg. by just downloading it from that repo) they are under no obligation to refrain from using it to compete with OpenAI. It is fair-use by the same argument OpenAI itself uses to train its own models on publicly available data.

My understanding is they legally can't. It was trained used OpenAI, which doesn't allow using their output to train new models. Someone would need to find another data source to fine tune llama.

sdrg822 4y ago
It’s only a matter of time
- hackernewds 4y ago
  waiting for the stable diffusion version of gpt still
dwallin 4y ago
You don't need to find a new data source, you just need to find an unencumbered third party. You can use the that data publicly provided in the git repo as long as you haven't signed an agreement with OpenAI yourself.
Zetice 4y ago
What will OpenAI do, sue? Okay but now it's out there.
- nwoli 4y ago
  Also how could that even be under protection. As if they haven’t been scraping copyrighted materials and sites with end user agreements to train the model in the first place
- TaylorAlexander 4y ago
  I believe the work was done by Stanford, so OpenAI could revoke Stanford's access to their API. That would inhibit Stanford's ability to do new research with this system.
- jerpint 4y ago
  OpenAI I knows they’re just a few cycles behind open source models - probably why they stuck a deal early with Microsoft.
hawski 4y ago
Why they can train their model on copyrighted data and claiming fair use as they do not outright copy while disallowing training other models on their output? I understand revoking access though.
- alwayslikethis 4y ago
  They realistically can not, I believe. If Microsoft is right that this is under fair use, then you are only limited by OpenAI ToS, which also says that the copyright of generated content belongs to the user. Do the following: Person A uses GPT3 to generate training data, and publish it on his blog, without representing it as human generated. Person A does not give permission for it to be used by Alpaca team. Alpaca team comes along, scrape his blog, and uses it as training data, without permission from person A. Now this is fair use, so there is nothing person A can do to stop it, just like how Github scraped our code for Copilot without permission.

simonw 4y ago

If they don't release the model, recreating it doesn't look too hard. $100 worth of compute time to run the fine-tuning, and the training data they used is here: https://github.com/tatsu-lab/stanford_alpaca/blob/main/alpaca_data.json https://github.com/tatsu-lab/stanford_alpaca/blob/main/alpac... That would have the same licensing problems that they have though: that alpaca_data.json file was created using GPT3. But creating a "clean" training set of 52,000 examples doesn't feel impossible to me for the right group.

dwallin 4y ago
You're only bound by the terms of OpenAI's agreement if you agreed to the terms of use. If a third party obtained the data without signing an agreement with OpenAI (eg. by just downloading it from that repo) they are under no obligation to refrain from using it to compete with OpenAI. It is fair-use by the same argument OpenAI itself uses to train its own models on publicly available data.