4 ms·
So we can SETI@Home or Folding@Home for large language models, now? Not sure how small the minimum size of the compute cluster can be. If it's still out of cons
by lucubratory 3y ago
So we can SETI@Home or Folding@Home for large language models, now? Not sure how small the minimum size of the compute cluster can be. If it's still out of consumer reach then this would either be just an intermediate research step, or a way for small-er (but still professional/well-resourced) labs to collaborate together. I'm not sure the latter would be helpful, as if they wanted to collaborate together they could probably already do that by pooling resources for a large cloud compute run.
- gaogao 3y agoThe limitations of the paper is more so maximum number of workers than minimum. It'd be pretty neat to get something that could work across 100+ distributed workers well.
- theendisney2 3y agoSomeone onhere argued this is entirely possible and that results can be merged later on. It seems facinating. Perhaps it is not conpute each node has to contribute but just bandwidth?