4 ms·
Question from an ML-illiterate: Are there known ways the training for these models could be distributed/decomposed? E.g. SETI-style distribution of a homogenou
by jeffparsons 4y ago
Question from an ML-illiterate:
Are there known ways the training for these models could be distributed/decomposed? E.g. SETI-style distribution of a homogenous centrally-defined task, or — much more exciting — recombination of several different models / sets of weights? (I'm just throwing words around here without really understanding them.)
I'm imaging a world in which one group of enthusiasts could work together to train a model on all images on Wikipedia, another group could work on training a model that understands hands really well, and then later yet another group could combine the work of the other two without doing all that training from scratch.
Is that even remotely plausible?
- sophrocyne 4y agoYes it is!
- swyx 4y agonot sure how illiterate you are, you're asking good questions, but fwiw if you watch the corridor digital video you should be able to grasp how much transfer learning is possible https://www.youtube.com/watch?v=W4Mcuh38wyM https://www.youtube.com/watch?v=W4Mcuh38wyM
- wokwokwok 4y agoIt’s almost certainly not worth the bother. The effort and time involved in setting up a distributed community training system would be extremely prone to abuse, errors and uncertainty about the results. You could get better quality, more quickly by simply running a kickstarter and paying for dedicated gpu time.