4 ms·
Wonder if someone would be willing to start an open source project where we could crowdsource donations for training, and people could possibly donate their GPU
by tarr11 4y ago
Wonder if someone would be willing to start an open source project where we could crowdsource donations for training, and people could possibly donate their GPU usage for it.
- polishdude20 4y agoThere's gotta be something like this already. Like a SETI @ Home type of thing .
- mryab 4y agoThere is! See https://petals.ml/ https://petals.ml/ for inference of models like BLOOM-176B over the internet or https://arxiv.org/abs/2301.11913 https://arxiv.org/abs/2301.11913 and https://arxiv.org/abs/2206.01288 https://arxiv.org/abs/2206.01288 that show you how to do pretraining from scratch in the same setting. Disclaimer: I'm a coauthor of these systems (including the one in OP)
- exebook 4y agoAmazing work! If I had a GPU, I'd join. I know similar project for text-to-image: https://aqualxx.github.io/stable-ui/ https://aqualxx.github.io/stable-ui/
- efreak 4y agohttps://github.com/aqualxx/stable-horde-notebook https://github.com/aqualxx/stable-horde-notebook My only problem with stable horde is that their anti-cp measure involves checking the prompt for words like small, meaning I can't use a nsfw-capable model with certain prompts (holding a very small bag, etc). That, and seeing great things in the image rating and being unable to reproduce because it doesn't provide the prompt.
- scottmf 4y agoThat’s what Stability AI has been doing… There are already open source LLMs the size of GPT-3 such as OPT and Bloom
- ipsum2 4y agoNo they haven't. Stability AI is funded by the founder and VC money, not crowdsourcing.