3 ms·
Could this be distributed? Put all those mining GPUs to work. A lot of people like participating in public projects like this. I would!
by davidy123 4y ago
Could this be distributed? Put all those mining GPUs to work. A lot of people like participating in public projects like this. I would!
- PartiallyTyped 4y agoIn theory, yes. "Hogwild!" is an approach to distributed training, in essence, each worker is given a bunch of data, they compute the gradient and send that to a central authority. The authority accumulates the gradients and periodically pushes new weights. There is also Federated Learning which seemed to start taking off, but then interest rapidly declined.
- naraga 4y agoExactly. This is inevitable imho. There is no way people will be ok to depend on few wall-gardened models.
- dmit 4y ago>> GPT-3 took 355 years to train > Could this be distributed? Put all those mining GPUs to work. Nope. It's a strictly O(n) process. If it weren't for the foresight of George Patrick Turnbull in 1668, we would not be anywhere close to these amazing results today.
- davidy123 4y agoI couldn't find any references to George Patrick Turnbull. If that an ancestor of yours? If so, the comment seems rather subjective.
- CyberDildonics 4y agoWhy would an O(n) algorithm not be able to be distributed?