4 ms·

They used OpenAI davinci to generate the dataset for instruction tuning. They mention one of the reasons this model is noncommercial is because OpenAI's terms o

They used OpenAI davinci to generate the dataset for instruction tuning. They mention one of the reasons this model is noncommercial is because OpenAI's terms of use prohibit developing models that compete with OpenAI. They also published the generated dataset. If one were to use this dataset to tune your own commercial model, would this be OK, or illegal? I would guess OpenAIs ToS no longer applies, as this is now Alpaca's dataset. You are never agreeing to OpenAIs ToS.

wongarsu 4y ago
I think it's a reasonable interpretation that Alpaca developers hold copyright over the dataset and can distribute it however they want, just how you hold copyright for any pictures you make with a camera. OpenAI can go after the Alpaca developers for the ToS violation, maybe force them to stop distributing the dataset on those grounds, but I don't see how that would affect third parties using that dataset.
freedmand 4y ago
The instruction tuning dataset is only 52,000 rows. It shouldn't be too hard to crowdsource high-quality human answers to this many questions and retrain the model, nixing the dependency on OpenAI.
- Tiberium 4y ago
  Such a thing already exists and there were some results - https://open-assistant.io https://open-assistant.io I'm not sure why the authors of Alpaca didn't try to train it on this dataset.
  - deleted 4y ago
    [deleted]
  - IanCal 4y ago
    That dataset isn't released yet. > Can I download the data? > You will be able to, under CC BY 4.0, but it's not released yet. We want to remove spam and PII before releasing it.
  - losteric 4y ago
    There's the OIG dataset (https://laion.ai/blog/oig-dataset/ https://laion.ai/blog/oig-dataset/) which was used to train a NeoX 20B ChatBot (https://huggingface.co/togethercomputer/GPT-NeoXT-Chat-Base-20B https://huggingface.co/togethercomputer/GPT-NeoXT-Chat-Base-...). The dataset is larger and publicly available. I want to try finetuning LLaMa on this tonight.
    - Jack5500 4y ago
      how did it go?
  - ilaksh 4y ago
    Wow.. I really hope someone will train this model with that dataset. Or maybe open assistant will pick it up. The results looks so promising.
scottlawson 4y ago
This model is for non commercial use only so it's unclear to me how it could compete commercially with OpenAI. Sure, you could decide to interpret "compete" in a way that makes it seem like this violates TOS, but the mom commercial restriction is a powerful counterargument too.
- vishal0123 4y ago
  They allow others to use it for noncommercial use. Other research groups won't have to use openAI APIs for some of the usecases, hence the model is competing.
hgsgm 4y ago
> OpenAI's terms of use prohibit developing models that compete with OpenAI. People shouldn't collaborate with Ope(ratio)nAI's attempt to poison the English language with its name. Government shouldn't allow lies to be trademarked brand names. It's fraud.

wongarsu 4y ago

I think it's a reasonable interpretation that Alpaca developers hold copyright over the dataset and can distribute it however they want, just how you hold copyright for any pictures you make with a camera. OpenAI can go after the Alpaca developers for the ToS violation, maybe force them to stop distributing the dataset on those grounds, but I don't see how that would affect third parties using that dataset.

freedmand 4y ago

The instruction tuning dataset is only 52,000 rows. It shouldn't be too hard to crowdsource high-quality human answers to this many questions and retrain the model, nixing the dependency on OpenAI.

Tiberium 4y ago
Such a thing already exists and there were some results - https://open-assistant.io https://open-assistant.io I'm not sure why the authors of Alpaca didn't try to train it on this dataset.
- deleted 4y ago
  [deleted]
- IanCal 4y ago
  That dataset isn't released yet. > Can I download the data? > You will be able to, under CC BY 4.0, but it's not released yet. We want to remove spam and PII before releasing it.
- losteric 4y ago
  There's the OIG dataset (https://laion.ai/blog/oig-dataset/ https://laion.ai/blog/oig-dataset/) which was used to train a NeoX 20B ChatBot (https://huggingface.co/togethercomputer/GPT-NeoXT-Chat-Base-20B https://huggingface.co/togethercomputer/GPT-NeoXT-Chat-Base-...). The dataset is larger and publicly available. I want to try finetuning LLaMa on this tonight.
  - Jack5500 4y ago
    how did it go?
- ilaksh 4y ago
  Wow.. I really hope someone will train this model with that dataset. Or maybe open assistant will pick it up. The results looks so promising.

scottlawson 4y ago

This model is for non commercial use only so it's unclear to me how it could compete commercially with OpenAI. Sure, you could decide to interpret "compete" in a way that makes it seem like this violates TOS, but the mom commercial restriction is a powerful counterargument too.

vishal0123 4y ago
They allow others to use it for noncommercial use. Other research groups won't have to use openAI APIs for some of the usecases, hence the model is competing.

hgsgm 4y ago

> OpenAI's terms of use prohibit developing models that compete with OpenAI. People shouldn't collaborate with Ope(ratio)nAI's attempt to poison the English language with its name. Government shouldn't allow lies to be trademarked brand names. It's fraud.