3 ms·
Your suggestion actually makes things much harder. A single “cycle” (and you need ~100-1,000+ cycles per second) would involve ~7,000 synapses per neuron * 86
by Retric 3y ago
Your suggestion actually makes things much harder.
A single “cycle” (and you need ~100-1,000+ cycles per second) would involve ~7,000 synapses per neuron * 86 billion neurons random memory accesses each second. Each of those accesses need to first read the location of memory and then access that memory.
For perspective a video card which is heavily optimized for this only gets you low billions of random memory access per second. True random access across large amounts of memory is hard. Of course if you’re using a 1 million GPU supercomputer you now need to sync all this memory between each GPU every cycle, or have each GPU handle network traffic for all these memory requests…
PS: The article assumes more efficient AI designs for good reason, actually using a neuronal network the size of the brain in real time is prohibative.
- aeonik 3y agoSuper computers are almost at this scale, if you go by FLOPS. Distributed cloud systems certainly surpass this number as well. https://www.top500.org/lists/top500/2023/11/ https://www.top500.org/lists/top500/2023/11/
- Retric 3y agoYea, FLOPS and RAM aren’t that problematic it’s really insane memory bandwidth or really novel hardware that’s at issue. That or some novel AI architecture. I doubt we need to actually emulate a brain to reach near parity with human cognition.