justheuristic

3 ms·

1. 1.▲
   HOGWILD! Inference – parallel LLM chain-of-thought with shared attention(huggingface.co)

   2 points by justheuristic 2y ago|0 comments
2. 2.▲
   by justheuristic 6y ago

   /* hypothesizing */ If you're using it for NLP, your dataset (token ids) typically weighs much less than intermediate tensors. So, i see two scenarios here: (1) distribute data chunks as you train using more conventional bitt
3. 3.▲
   Learning@Home - decentralized training of huge neural networks(learning-at-home.github.io)

   5 points by justheuristic 6y ago|1 comments
4. 4.▲
   by justheuristic 6y ago

   Can you train a huge neural network without a supercomputer? Imagine you want a GPT-3-sized model, but instead of $10⁸ GPU cluster you've got support from thousands of volunteers across the world - gamers, research labs, small companie
5. 5.▲
   A GitHub course in reinforcement learning in the wild(github.com)

   2 points by justheuristic 9y ago|0 comments