There's Petals[0], but the problem seems to be that the entire training data needs to be loaded into VRAM and can't be split up across devices. [0] https://git

There's Petals[0], but the problem seems to be that the entire training data needs to be loaded into VRAM and can't be split up across devices. [0] https://github.com/bigscience-workshop/petals https://github.com/bigscience-workshop/petals