The DeepMind paper on synthetic gradients is super interesting for training in clusters. I hope it gets built into TF soon. Also really hope there will be more

The DeepMind paper on synthetic gradients is super interesting for training in clusters. I hope it gets built into TF soon. Also really hope there will be more options for GPU hardware on public clouds. Eventually the pieces will come together and it will be trivial to deploy cloud 400gpus for an hour to run the load your local 4gpu workstation would spend 100 hours on, but we are definitely not there yet. We are working on a packaging format called snappy (http://www.snapcraft.io http://www.snapcraft.io) and starting to talk to TF guys about packaging TF with it (kubernetes is already packaged as a snap) - hopefully this will take some pain away once it's working