5 ms·
I understood SETI style meaning crowdsourced. Instead of mining bitcoin you mine LLMs. It's a nice idea I think. Not sure about technical details, bandwidth lim
by lz400 2y ago
I understood SETI style meaning crowdsourced. Instead of mining bitcoin you mine LLMs. It's a nice idea I think. Not sure about technical details, bandwidth limitations, performance, etc.
- Mountain_Skies 2y agoSETI had a clear purpose that donors of computer resources could get behind. The LLM corps early on decided to drink the steering poison that will keep there from ever being a united community for making open LLMs. At best you'll get a fractured world of different projects, each with its own steering directives.
- Aerroon 2y agoThe internet is for ____. That could be a factor that unites enough people to donate their compute time to build diffusion models. At least if it was easy enough to set up.
- CaptainFever 2y agoRelated: people donating computing power to run diffusion and text models, which is definitely largely used for porn. https://stablehorde.net/ https://stablehorde.net/ Or the large amounts of community efforts (not exactly crowd sourced though) for diffusion fine-tunes and tools! Pony XL, and other uncensored models, for example. I haven't kept up with the rest, because there's just too much.
- lostmsu 2y agoYou don't have to donate, we will pay you for idle time of your gaming GPU: https://borg.games/setup https://borg.games/setup
- HPsquared 2y agoUnfortunately, LLM training is not as computationally easy (embarrassingly parallel) as mining bitcoins.
- lz400 2y agodamn it! but nice research area
- ghxst 2y agoIf that were to be solved (if at all possible, and feasible / competitive) I can definitely see "LLM mining" be a historic milestone. Also much closer to the spirit of F@H in some sense, depending how you look at it. Would there be a financial incentive? And how would it be distributed? Could you receive a stake in the LLM proportional to the contribution you did? Would that be similar in some sense to purchasing stock in an AI company, or mining tokens for a crypto currency? Potentially a lot of opportunity here.
- HPsquared 2y agoThe models are too large to fit on a desktop GPU's VRAM. Progress would either require smaller models (MoE might help here? not sure) or bigger VRAM. For example training a 70 billion parameter model would require at least 140GB of VRAM in each system, whereas a large desktop GPU (4090) has only 24GB. You need enough memory to run the unquantized model for training, then stream the training data through - that part is what is done in parallel, farming out different bits of training data to each machine.
- mr_toad 2y agoData parallel training is not the only approach. Sometimes the model itself needs to be distributed across multiple GPU. https://www.microsoft.com/en-us/research/blog/zero-deepspeed-new-system-optimizations-enable-training-models-with-over-100-billion-parameters/ https://www.microsoft.com/en-us/research/blog/zero-deepspeed... The communications overhead of doing this over the internet might be unworkable though.