3 ms·
Yes there is petals/bloom https://github.com/bigscience-workshop/petals https://github.com/bigscience-workshop/petals but it's not so great. Maybe it will impro
by ftxbro 4y ago
Yes there is petals/bloom https://github.com/bigscience-workshop/petals https://github.com/bigscience-workshop/petals but it's not so great. Maybe it will improve or a better one will come.
- whalesalad 4y agoReally interesting live monitor of the network: http://health.petals.ml http://health.petals.ml
- polishdude20 4y agoI wonder how they handle illegal content. Like, if you're running training data on your computer, what's to stop someone else's data that is illegal, from being uploaded to your computer as part of training?
- riedel 4y agoI read that it is only scoring the model collaboratively but it allows some fine-tuning I guess. Getting the actual gradient descent to parallelize is more difficult because one needs to average the gradient when using data/batch parallelism. It becomes more a network speed than GPU speed problem. Or are LLMs somehow different?