This is very interesting. I was very surprised that Tensorflow did not use a central parameter server, and this looks like a good foundational parameter server

This is very interesting. I was very surprised that Tensorflow did not use a central parameter server, and this looks like a good foundational parameter server to build something like DistBelief on top of from Microsoft. I wonder if they have a Deep Learning module that they will open source eventually. Microsoft has a few tricks they used in some of their recent papers and it would be interesting to see them in a production quality system.