While this is interesting for TensorFlow, I think that it will not result in more than an evolutionary step forward in AI. The reason being that the single gre

While this is interesting for TensorFlow, I think that it will not result in more than an evolutionary step forward in AI. The reason being that the single greatest performance boost for computing in recent memory was the data locality metaphor used by MapReduce. It lets us get around CPU manufacturers sitting on their hands and the fact that memory just isn’t going to get substantially faster. I'd much rather see a general purpose CPU that uses something like an array of many hundreds or thousands of fixed-point ALUs with local high speed ram for each core on-chip. Then program it in a parallel/matrix language like Octave or as a hybrid with the actor model from Erlang/Go. Basically give the developer full control over instructions and let the compiler and hardware perform those operations on many pieces of data at once. Like SIMD or VLIW without the pedantry and limitations of those instruction sets. If the developer wants to have a thousand realtime linuxes running Python, then the hardware will only stand in the way if it can’t do that, and we’ll be left relying on academics to advance the state of the art. We shouldn’t exclude the many millions of developers who are interested in this stuff by forcing them to use notation that doesn’t build on their existing contextual experience. I think an environment where the developer doesn’t have to worry about counting cores or optimizing interconnect/state transfer, and can run arbitrary programs, is the only way that we’ll move forward. Nothing should stop us from devoting half the chip to gradient descent and the other half to genetic algorithms, or simply experiment with agents running as adversarial networks or cooperating in ant colony optimization. We should be able to start up and tear down algorithms borrowed from others to solve any problem at hand. But not being able to have that freedom - in effect being stuck with the DSP approach taken by GPUs, is going to send us down yet another road to specialization and proprietary solutions that result in vendor lock-in. I’ve said this many times before and I’ll continue to say it as long as we aren’t seeing real general-purpose computing improving.