3 ms·
The main highlight is a new version of AllReduce native to TensorFlow. This, like Horovod, builds a ring and has both native and NCCL implementations. Early res
by jamesblonde 8y ago
The main highlight is a new version of AllReduce native to TensorFlow. This, like Horovod, builds a ring and has both native and NCCL implementations. Early results (albeit on a single host) show that it outperforms Horovod:
https://groups.google.com/a/tensorflow.org/forum/#!topic/discuss/7T05tNV08Us https://groups.google.com/a/tensorflow.org/forum/#!topic/dis...
- minimaxir 8y agoAnd it looks like it's natively supported via Keras too? https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/distribute https://github.com/tensorflow/tensorflow/tree/master/tensorf...
- jamesblonde 8y agoYes, and with the estimator API, too.