Inplace is a nice late phase optimization, but the root problem here appears to be that Google's convolution kernels are crap compared to those in cuDNN3 and Ne

Inplace is a nice late phase optimization, but the root problem here appears to be that Google's convolution kernels are crap compared to those in cuDNN3 and Neon (I asked Scott Gray about this directly and I trust his wisdom). No surprises there whatsoever. The tensorflow engine is the most gold-plated POS I've seen in a long time. if it were running on 1000+ servers, I'd get the level of overengineering they've applied here. But single server pthreads? WTF? Also parameter server is dumb unless you sweat the implementation of the gather/reduction ops, but I digress.