There is Petals [0] which is basically what you're asking about: "Run large language models at home, BitTorrent‑style." It doesn't include training from scratc

There is Petals [0] which is basically what you're asking about: "Run large language models at home, BitTorrent‑style." It doesn't include training from scratch, but it supports fine-tuning and inference powered by a swarm of GPUs. [0] https://petals.dev/ https://petals.dev/