5 ms·
Moore's law (or at least, the diminishing one) is not relevant here because these are not single threaded programs. Google put 8x on their TPUv2 -> v3 upgrade;
by nmca 8y ago
Moore's law (or at least, the diminishing one) is not relevant here because these are not single threaded programs. Google put 8x on their TPUv2 -> v3 upgrade; parallel matrix multiplies at reduced precision are a long way away from any theoretical limits, as I understand it.
- jacksmith21006 8y agoTotally agree but why on earth down voted? The first generation TPUs used 65536 very simple cores. In the end you have so many transistors you can fit and there are options on how to arrange and use. You might support very complex instructions and data types and then four cores. Or you might only support 8 bit ints, very, very simple instructions and use 65536 cores. In the end what matters is the joules to get something done. We can clearly see that we have big improvements by using new processor architectures.