4 ms·

Major kudos to Mistral for being the first company to Apache license a model of this class. Meta wouldn't make LLama open source. DeciLM wouldn't make theirs

Major kudos to Mistral for being the first company to Apache license a model of this class. Meta wouldn't make LLama open source. DeciLM wouldn't make theirs open source. All of them wanted to claim they were open source, while putting in place restrictions and not using an open source license. So I think it's worth giving Mistral big kudos here for actually doing it and going Apache licensed.

miloignis 3y ago
Falcon 40B is Apache2, though they then went back to not open for their 180B.
- divbzero 3y ago
  MPT-30B is also Apache 2.0: https://huggingface.co/mosaicml/mpt-30b https://huggingface.co/mosaicml/mpt-30b There must be others as well?
  - capableweb 3y ago
    A lot! Go here https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderb..., check "Hub license" on the left then sort by that column in the table. Estimating, there is more than 100 models with a apache-2.0 license.
    - avereveard 3y ago
      a good chunk are "only the lora is apache 2 the base model is lama" or straight up llama2 model finetuned and distributed laundring their license, or use gpt generated code in the finetuning dataset against OpenAI tos. licensing is a bit up in the air and just filtering with apache 2 doesn't tell you much
blueboo 3y ago
Persimmon-8B from Adept did it first no https://www.adept.ai/blog/persimmon-8b https://www.adept.ai/blog/persimmon-8b
dartos 3y ago
Have you seen the RWKV model? They have a 40B one and IIRC are part of the Linux foundation now too
ekelsen 3y ago
That would be Persimmon-8B, no? https://www.adept.ai/blog/persimmon-8b https://www.adept.ai/blog/persimmon-8b
logicchains 3y ago
> Major kudos to Mistral for being the first company to Apache license a model of this class. Falcon 40B is under the Apache license, and as a 40B model it outperforms this 7B model on most tasks.

miloignis 3y ago

Falcon 40B is Apache2, though they then went back to not open for their 180B.

divbzero 3y ago
MPT-30B is also Apache 2.0: https://huggingface.co/mosaicml/mpt-30b https://huggingface.co/mosaicml/mpt-30b There must be others as well?
- capableweb 3y ago
  A lot! Go here https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderb..., check "Hub license" on the left then sort by that column in the table. Estimating, there is more than 100 models with a apache-2.0 license.
  - avereveard 3y ago
    a good chunk are "only the lora is apache 2 the base model is lama" or straight up llama2 model finetuned and distributed laundring their license, or use gpt generated code in the finetuning dataset against OpenAI tos. licensing is a bit up in the air and just filtering with apache 2 doesn't tell you much

blueboo 3y ago

Persimmon-8B from Adept did it first no https://www.adept.ai/blog/persimmon-8b https://www.adept.ai/blog/persimmon-8b

dartos 3y ago

Have you seen the RWKV model? They have a 40B one and IIRC are part of the Linux foundation now too

ekelsen 3y ago

That would be Persimmon-8B, no? https://www.adept.ai/blog/persimmon-8b https://www.adept.ai/blog/persimmon-8b

logicchains 3y ago

> Major kudos to Mistral for being the first company to Apache license a model of this class. Falcon 40B is under the Apache license, and as a 40B model it outperforms this 7B model on most tasks.