There are methods that make it feasible to train models over the internet. DiLoCo is one [1] and NousResearch has found a way to improve on that using a method

There are methods that make it feasible to train models over the internet. DiLoCo is one [1] and NousResearch has found a way to improve on that using a method they call DisTro [2]. 1. https://arxiv.org/abs/2311.08105 https://arxiv.org/abs/2311.08105 2. https://github.com/NousResearch/DisTrO?tab=readme-ov-file https://github.com/NousResearch/DisTrO?tab=readme-ov-file