4 ms·

I hear you on the parallelism, the flip side is that each neuron doesn’t need anywhere near the flexibility of a general purpose CPU. I think many of the AI al

I hear you on the parallelism, the flip side is that each neuron doesn’t need anywhere near the flexibility of a general purpose CPU. I think many of the AI algorithms can run on GPUs, where core counts can get into the thousands (still very far from millions). But these are largely independent — the interconnects between GPUs are not on the scale of neurons. My guess: we develop specialized hardware that, over time, will increasingly resemble organic brain structure.

0815test 7y ago
"Core counts" on GPUs are quite misleading - what are called "cores" on GPU hardware would be called "execution units/ALU's" on the CPU. GPUs have significantly-enhanced memory throughput, and can use a combination of SIMD (with masked/predicated instructions), barrel processing and genuine multicore to excel at embarrassingly-parallel compute. But they're not magic. And if what you care about is running a wide variety of mainstream stats or machine learning algorithms, the "specialized hardware" you mention is the modern GPU, or something very much like it - I don't see any low-hanging fruit that might make some other HW architecture more feasible. Fixed-point compute might get there someday, but it's way too fiddly at present - it's not really clear how to "tweak" mainstream learning algorithms so as to dispense with the large dynamic range that floating point compute provides.