6 ms·
It's only single node, multiple GPUs, though, which already exists in other frameworks.
by brianchu 11y ago
It's only single node, multiple GPUs, though, which already exists in other frameworks.
- limau 11y agoWith the parallelism model and abstraction it has to support model/data or hybrid partitioning, and synchronous/asynchronous or hybrid training, it should be easy to extend to GPU cluster. However, training is required only periodically, and if it can be done on existing clusters as efficiently, why not?