3 ms·

> one operation per clock cycle […] Modern CPUs are so many orders of magnitude slower than they could be with a parallel architecture Modern CPUs are supersca

> one operation per clock cycle […] Modern CPUs are so many orders of magnitude slower than they could be with a parallel architecture Modern CPUs are superscalar, meaning they can execute multiple operations per clock cycle. > processors that were 3/4 cache memory, with barely any transistors used for logic. It's surely worse than that now, with the vast majority of logic gates on chips just sitting around idle That's because memory is the bottleneck for 99% of compute workloads. The only things that memory is not the bottleneck for is silly arithmetic (e.g. graphics) and network processing. And we have processors for those workloads. (Source: I've programmed Tilera network processors.) > even with OpenCL/CUDA because it frankly sucks to do any real programming when all you have at your disposal is DSP concepts. OpenCL/CUDA are so unlike DSP programming it's not even funny. SIMT and SIMD are very different programming models. (Source: I've programmed both CUDA and DSPs (Blackfin specifically).) > it’s time to let go of VHDL/Verilog because it’s one level too low. We really need a language above them that lets us wire up basic logic without fear of the chip burning up. Huh? VHDL and Verilog are very high-level languages that expose concepts like open-drain/collector logic and different voltage domains only unwillingly. Unless you have a really crappy elaborator, "chips burning up" just ain't gonna happen. (Source: I've done mixed-signal ASIC design.) > So instead of wasting vast swaths of the chips for something stupid like cache, it can go to storage for logic layouts. I don't think you understand the importance of cache. Memory is S-L-O-W. (Source: I've hand-optimized high-performance network processing code.) > AI, physics simulations, formula discovery, protein folding, basically all of the problems that current single threaded architectures can’t touch in a cost-effective manor All those things you list are bottlenecked by memory bandwidth, not CPU time.

zackmorris 12y ago
I just meant that OpenCL/CUDA are moving towards a narrower scope than what I would like. There is too much emphasis on manually arranging your data with the correct interleaving, or doing things in batches, pipelined in just the right order. It all reminds me of DSP, which I was never a huge fan of. That's because all of the supposed gains can easily be replicated with a general purpose language and an order of magnitude more computing power (which is easily attainable with billions of transistors at your disposal, getting cheaper every year). You are right about memory bandwidth though. I'm excited that memory has come so far, with wide channels running at a GHz or more. But, that has come at the expense of latency. I think this problem is intractable, because everything is moving toward data centers with even slower interconnect. I’m reaching here, but I think the endgame will be memory that works more like bittorrent, where the latency won’t be how far the nearest neighbor is, but how much the data differs from what you already have. Then we can eliminate most of the problems with slow interconnect and move to a big dumb hash system that uses diffs but results in orders of magnitude higher throughput. If processing is really not a bottleneck, then this will be trivial.
- colanderman 12y ago
  I think what's more likely is that processing will move into the memory itself. We already see this somewhat with manycore shared-cache architectures like Tilera, which effectively provide one core per ¼ MiB of L3 cache. (Though unfortunately L3 cache latency is only ~twice as fast as RAM, but at least the bandwidth is much better.)

zackmorris 12y ago

I just meant that OpenCL/CUDA are moving towards a narrower scope than what I would like. There is too much emphasis on manually arranging your data with the correct interleaving, or doing things in batches, pipelined in just the right order. It all reminds me of DSP, which I was never a huge fan of. That's because all of the supposed gains can easily be replicated with a general purpose language and an order of magnitude more computing power (which is easily attainable with billions of transistors at your disposal, getting cheaper every year). You are right about memory bandwidth though. I'm excited that memory has come so far, with wide channels running at a GHz or more. But, that has come at the expense of latency. I think this problem is intractable, because everything is moving toward data centers with even slower interconnect. I’m reaching here, but I think the endgame will be memory that works more like bittorrent, where the latency won’t be how far the nearest neighbor is, but how much the data differs from what you already have. Then we can eliminate most of the problems with slow interconnect and move to a big dumb hash system that uses diffs but results in orders of magnitude higher throughput. If processing is really not a bottleneck, then this will be trivial.

colanderman 12y ago
I think what's more likely is that processing will move into the memory itself. We already see this somewhat with manycore shared-cache architectures like Tilera, which effectively provide one core per ¼ MiB of L3 cache. (Though unfortunately L3 cache latency is only ~twice as fast as RAM, but at least the bandwidth is much better.)