I've been surviving a winter of my own since 1997 when video cards took over and Moore's Law for CPUs started slowing down, finally ending around 2007 when smar

I've been surviving a winter of my own since 1997 when video cards took over and Moore's Law for CPUs started slowing down, finally ending around 2007 when smartphones arrived and research dollars got redirected to cost and power reduction. We also got stuck with stopgap SIMD measures like SSE/AltiVec instead of a truly scalable multicore/multiprocessor MIMD architecture. Which made it much more difficult to experiment with stuff like genetic algorithms or any other machine learning algorithm that don't map well to vectors. Because a 10 billion transistor GPU that reports thousands of shading units may only have 64 physical cores. Whereas something like a 1 GHz DEC Alpha with 15 million transistors would give us more on the order of 100-1000 cores on the same die, especially if the cache was removed and it used a distributed content-addressable memory for data locality and a near-linear bandwidth increase for workloads like map-reduce. Anyway, maybe something like MIMD could be built on GPGPU? I just want to get info on my system and see 1024+ threads and 10 TB/sec memory bandwidth on a chip that costs no more than $1000. The fact that this doesn't exist today after all these years, and that people are perfectly happy to keep repurchasing the same desktop computer the same speed as ones from a decade ago, and on top of that go to the ends of the earth to get the simplest shader code working (rather than just coding directly in stuff like Rust/D/C#/Java/C++ and Docker and/or Lisp) just absolutely blows my mind. https://en.wikipedia.org/wiki/Multiple_instruction,_multiple_data https://en.wikipedia.org/wiki/Multiple_instruction,_multiple... https://en.wikipedia.org/wiki/Transputer https://en.wikipedia.org/wiki/Transputer http://aggregate.org/MOG/ http://aggregate.org/MOG/ https://www.microsoft.com/en-us/research/video/mimd-on-gpu/ https://www.microsoft.com/en-us/research/video/mimd-on-gpu/