3 ms·
Yep, that and SETI@Home. I loved the eye candy, even if I didn't know what it fully meant.
by _joel 1y ago
Yep, that and SETI@Home. I loved the eye candy, even if I didn't know what it fully meant.
- seydor 1y agoHow come we don't have AI@Home
- throwup238 1y agoThe network bandwidth between nodes is a bigger limitation than compute. The newest Nvidia cards come with 400gbit busses now to communicate between them, even on a single motherboard. Compared to SETI or Folding @Home, this would work glacially slow for AI models.
- fourthark 1y agoSeems like training would be a better match, where you need tons of compute but don’t care about latency.
- ronsor 1y agoNo, the problem is that with training, you do care about latency, and you need a crap-ton of bandwidth too! Think of the all_gather; think of the gradients! Inference is actually easier to distribute.
- meehai 1y agoYeah, but if you can do topologies based on latencies you may get some decent tradeoffs. For example with N=1M nodes each doing batch updates in a tree manner, i.e the all reduce is actually layered by latency between nodes.
- shaklee3 1y ago800Gbps
- deleted 1y ago[deleted]
- gregsadetsky 1y agoThat and project RC5 from the same time period..! :-) https://www.distributed.net/RC5 https://www.distributed.net/RC5 https://en.wikipedia.org/wiki/RSA_Secret-Key_Challenge https://en.wikipedia.org/wiki/RSA_Secret-Key_Challenge I wonder what kind of performance would I get on a M1 computer today... haha EDIT: people are still participating in rc5-72...?? https://stats.distributed.net/projects.php?project_id=8 https://stats.distributed.net/projects.php?project_id=8