3 ms·
At least 1) debugging in a numpy-like env and 2) multi GPU training on a single machine with multiple GPU (last time I was using TF it was such a nightmare that
by CretinDesAlpes 8y ago
At least 1) debugging in a numpy-like env and 2) multi GPU training on a single machine with multiple GPU (last time I was using TF it was such a nightmare that people had to use horovod, a wrapper library developed by... Uber).
- me2too 8y agoPoint 1 is a good point, although tfdebug exists and it works. For point 2: I don't see the problem. I use almost daily a wrapper (defined inside tf.contrib, that in version 2.x will go in core [I hope]) around the optimizer that in 2 lines allows me to distribute the training on multiple GPUs on the same machine
- jamesblonde 8y agoFor a comparison of Horovod and TensorFlow's new CollectiveAllReduce, read here: https://www.logicalclocks.com/goodbye-horovod-hello-tensorflow-collectiveallreduce/ https://www.logicalclocks.com/goodbye-horovod-hello-tensorfl...