3 ms·
Google unveils way to train AI models across distributed data centers
- gnabgib 7mo agoSource: https://storage.googleapis.com/deepmind-media/DeepMind.com/Blog/decoupled-diloco-a-new-frontier-for-resilient-distributed-ai-training/decoupled-diloco-for-resilient-distributed-pre-training.pdf https://storage.googleapis.com/deepmind-media/DeepMind.com/B... (https://news.ycombinator.com/item?id=47897325 https://news.ycombinator.com/item?id=47897325)
- SilverElfin 7mo agoIs this actually innovative? I respect there’s a lot of work in making it reality and doing it specifically for AI training. But doing portions of work in clusters that are far apart and combining them has been done many times before for non AI things, right? Or so I would think.