Actually there is the Petals [0] project which does that (I think it's more for inference and fine-tuning, not full training) and might be the best current appr

Actually there is the Petals [0] project which does that (I think it's more for inference and fine-tuning, not full training) and might be the best current approach for "self-hosted LLM", though it defeats the point of privacy/anonymity since everyone in the distributed swarm have access to your data [0] https://github.com/bigscience-workshop/petals https://github.com/bigscience-workshop/petals