Computer Organization and Design by Patterson and Hennessy is still sitting on my shelf, saved from my computer engineering classes at UIUC. It's probably the

Computer Organization and Design by Patterson and Hennessy is still sitting on my shelf, saved from my computer engineering classes at UIUC. It's probably the closest thing I have to a definitive source on the hard limits faced in processor design. IMHO since I graduated in 1999 there has been almost no innovation in processor cores, and in fairness, maybe there couldn’t have been (as the limits were well understood by then). Instead the industry opted to go with smaller and smaller features which led to skyscraper interconnects that act as much like transmission lines as wires, with deep pipelines and large areas of the chip devoted to branch prediction, large percentages of chip area devoted to cache, niche processing units like GPUs on-chip rather than general purpose DSP for multiprocessing, obfuscated microcode rather than RISC, just layers and layers of cruft that were driven more by demands for single-threaded performance and low power consumption than innovation for its own sake. So it shouldn’t be any real surprise that Moore’s law died somewhere around 2005, give or take. Computers today are still the 3 MHz they were then, transistor counts have gone up but real-world performance has increased mostly due to faster DDR and SSD, not faster cores. To get an idea of how far behind CPUs have fallen, just look at the throughput of any video card vs, say, a Core i7. We’re talking 3 orders of magnitude better performance for GPUs, about what a decade and a half of compounded transistor count for a multicore architecture would predict. So it saddens me that there never really were other competitors to CISC for consumers. I wonder if something like an ARM or AMD with a MIPS instruction set aiming for more cores like the Teraflops or Kilocore that ran OpenGL (and TensorFlow and other AI libs had they existed) natively might have done for computing back in the day. But it was a strange time in history after the Dot Bomb. The carpet got pulled out of research at a time when it was perhaps needed most. Everyone had to pull themselves up by their bootstraps and we all became entrepreneurs instead of engineers. Maybe that’s a good thing, maybe it’s not, I don’t really know. But I do know that we’ve had about a 15 year setback in computing and we are roaming the desert of informal methods on some kind of walkabout or vision quest to see where it’s all headed. Personally, I don’t see any future here if we don’t fix the fundamental problems in patents and intellectual property. It’s fun and everything to be hip about technology but at the end of the day someone has to push the state of the art forward. I know precisely what I want: as many general purpose cores as possible running as fast as possible with a self-organizing interconnect to as many other computers as possible so I can run the simulations I thought we’d be running by now. I want a stream to that computer, no fancy libraries in my way, no proprietary OS. I want to program it with MATLAB, Erlang, Go, Clojure Haskell, Scala, etc etc etc. I don’t want CUDA, I don’t want to have to lease time from the cloud, I don’t want to have to depend on the largess of billion dollar companies’ research programs to innovate in AI. I may have forgotten the details of how to achieve these things but I haven’t forgotten the original dream of ever-cheaper transistors giving every person access to a supercomputer and eventually R2D2. We’ve scaled data but we simply haven’t scaled processing.