2 ms·
Ask HN: ML models trained on decentralized computing?
Could models such as Dalle2 (huge amount of parameters), be trained on distributed/decentralized network of volunteer/paid computers in the style of LHC at home [0]
[0] https://en.wikipedia.org/wiki/LHC@home
- verdverm 4y agowhat does backprop look like? how long does a single training iteration take? what happens if someone drops out mid loop? how much does it cost to move the training data around?