2 ms·
The problem is Amdahl's law. You can only parallelize so much. While the brain is certainly extremely parallelized, neural nets do not employ the same algorithm
by pron 9y ago
The problem is Amdahl's law. You can only parallelize so much. While the brain is certainly extremely parallelized, neural nets do not employ the same algorithms as the brain, and so, unless we find algorithms that are more amenable to parallelization, Amdahl's law is going to get us.
- JacobiX 9y agoMost modern neural networks implementations are parallelized. And that is why we can run them extremely well on the GPU. For example Volta GPUs delivers 5X increase in deep learning training compared to prior generation NVIDIA Pascal architecture. This is why I was asking for clarification about the hardware claims.
- pron 9y agoOf course they are, but they're not as parallelizable as the brain, which is why they're subject to Amdahl's law.