I just meant that OpenCL/CUDA are moving towards a narrower scope than what I would like. There is too much emphasis on manually arranging your data with the c

I just meant that OpenCL/CUDA are moving towards a narrower scope than what I would like. There is too much emphasis on manually arranging your data with the correct interleaving, or doing things in batches, pipelined in just the right order. It all reminds me of DSP, which I was never a huge fan of. That's because all of the supposed gains can easily be replicated with a general purpose language and an order of magnitude more computing power (which is easily attainable with billions of transistors at your disposal, getting cheaper every year). You are right about memory bandwidth though. I'm excited that memory has come so far, with wide channels running at a GHz or more. But, that has come at the expense of latency. I think this problem is intractable, because everything is moving toward data centers with even slower interconnect. I’m reaching here, but I think the endgame will be memory that works more like bittorrent, where the latency won’t be how far the nearest neighbor is, but how much the data differs from what you already have. Then we can eliminate most of the problems with slow interconnect and move to a big dumb hash system that uses diffs but results in orders of magnitude higher throughput. If processing is really not a bottleneck, then this will be trivial.