This has definitely been discussed. There have even been some projects, although I haven't checked on the status of any of them lately. As best as I can recall,

This has definitely been discussed. There have even been some projects, although I haven't checked on the status of any of them lately. As best as I can recall, there are some specific structural reasons why it's hard to train LLM's this way, but I don't recall all the details offhand. https://www.google.com/search?q=distributed+model+training+site%3Anews.ycombinator.com https://www.google.com/search?q=distributed+model+training+s... https://news.ycombinator.com/item?id=35799843 https://news.ycombinator.com/item?id=35799843 https://www.reddit.com/r/ArtificialInteligence/comments/18q6t8t/stupid_question_but_is_anyone_attempting_to_use/ https://www.reddit.com/r/ArtificialInteligence/comments/18q6... https://www.reddit.com/r/slatestarcodex/comments/1gtnxgd/what_is_needed_to_allow_berkeley_boinc_or_similar/ https://www.reddit.com/r/slatestarcodex/comments/1gtnxgd/wha... https://github.com/BOINC/boinc/wiki/Using-BOINC-for-AI https://github.com/BOINC/boinc/wiki/Using-BOINC-for-AI https://the-decoder.com/ai-startup-prime-intellect-trains-first-distributed-llm-across-three-continents/ https://the-decoder.com/ai-startup-prime-intellect-trains-fi...