13 ms·
Moore's Law (on a single CPU core) effectively ended around 2009. Our programming idioms for parallel processing are still so crude, hardly anyone can write mai
by nickbauman 10y ago
Moore's Law (on a single CPU core) effectively ended around 2009. Our programming idioms for parallel processing are still so crude, hardly anyone can write maintainable parallelized code for anything but toy problems. Until we can restart Moore's Law again, this is all academic.
http://www.globalnerdy.com/2007/09/07/multicore-musings/ http://www.globalnerdy.com/2007/09/07/multicore-musings/
- argonaut 10y agoTo be fair, many machine learning algorithms (especially deep learning) are very parallelizable, both when training and predicting.
- grkvlt 10y agoBut the law says nothing about cores or clock speeds etc. just transistors and their sizes. The regular reductions in feature size on dies are still happening, as I understand it.
- nojvek 10y agoSize of copper atom is 0.2nm. We are already at 10nm fab processes. We can't really get too small otherwise it gets unstable and heat becomes a major issue. On the positive side, electric flows much faster than synapses and transistors are smaller than neurons. So with today's technology if we were to pack silicon parallel cores in size of human brain we'd have a much faster brain.
- hetid 10y agoThe heat issue makes the latter impossible.
- joefourier 10y agoToy problems? GPGPU is widely used in real-world applications such as image processing, compression, computer graphics, physical simulations, cryptography and machine learning, which are all embarrassingly parallel. With regards to programming, I would not all CUDA and OpenCL crude at all.
- nickbauman 10y agoInteresting in that almost all those problems are share-nothing, i.e. no shared state.