5 ms·

175B is for research only and as far as I understood their ToU does not allow commercial usage. Currently, the largest LLM that is both free and commercially u

by option 4y ago

175B is for research only and as far as I understood their ToU does not allow commercial usage. Currently, the largest LLM that is both free and commercially usable (Apache 2.0) is 100B YaLM from Yandex (russian’s copy of Google). However, they did not publish any details on their training data.

plegresl 4y ago
The 176B parameter BLOOM model should be available soon: https://bigscience.notion.site/BLOOM-BigScience-176B-Model-ad073ca07cdf479398d5f95d88e218c4 https://bigscience.notion.site/BLOOM-BigScience-176B-Model-a...
- option 4y ago
  yes, looking forward to it especially because it is going to be multilingual by design
charcircuit 4y ago
>However, they did not publish any details on their training data. Yes they did. It's in the README.
- option 4y ago
  All I can see is “ 1.7 TB of online texts, books, and countless other sources in both English and Russian.” if there are more details, can you please share a link? I am worried that “other sources” may contain Yandex.news which is a cesspool of anti-West and anti-Ukraine propaganda
  - 533474 4y ago
    The pile dataset is used for the English language
- remram 4y ago
  Direct link: https://github.com/yandex/YaLM-100B/blob/main/README.md#training-details https://github.com/yandex/YaLM-100B/blob/main/README.md#trai...
timmg 4y ago
Dumb question: does 175B parameters mean the number of bytes (or floats?) in the model? Does that also mean you need the whole model in memory to do inference (in practice)? If so, not many machines have that much RAM. Makes it hard to "play" with.
- lostmsu 4y ago
  float16s or bfloat16s so 2x of that for storage You can infer using DeepSpeed.