3 ms·
In terms of quality, I think BLOOMZ, or mT0, are the best open-source ones. The non-finetuned BLOOM does not appear favorably (in English) compared to GLM or O
by espadrine 4y ago
In terms of quality, I think BLOOMZ, or mT0, are the best open-source ones.
The non-finetuned BLOOM does not appear favorably (in English) compared to GLM or OPT, which both have published weights: https://crfm.stanford.edu/helm/v0.1.0/?group=mmlu https://crfm.stanford.edu/helm/v0.1.0/?group=mmlu and Flan-T5 is above OPT-IML: https://arxiv.org/pdf/2212.12017.pdf https://arxiv.org/pdf/2212.12017.pdf
> Is the future going to be controlled by big corporations who own the models themselves?
On this subject, there is an effort stemming from BigScience to build an open, distributed inference network, so that people that don’t have enough GPUs at home can contribute theirs and get text generation at one word per second: https://github.com/bigscience-workshop/petals#how-does-it-work https://github.com/bigscience-workshop/petals#how-does-it-wo...