4 ms·
As you say, it is tricky. Still, the current trend is that TPU's are taking over for CUDA-like GPU's for solving AI problems (such as with Alpha0), so it could
by trashtester 8y ago
As you say, it is tricky. Still, the current trend is that TPU's are taking over for CUDA-like GPU's for solving AI problems (such as with Alpha0), so it could be argued that tensor operations is the most relevant metric for estimating when human level AI will be possible.
Of course, tensor operations do not map 1:1 vs FLOPS, even for AI problems. On the other hand, tensor oriented hardware is currently getting faster at a higher rate than traditional GPU's.
But RK's prediction did assume that we would have reached full 3d chip manufacturing processes by now, and we may not be able to have human brain levels of performance on a single die until we do, which looks like it will take 5-10 years.
- jcranmer 8y agoTo be clear about one thing: When LINPACK measures FLOPS, it doesn't count how many instructions the computer executed. It uses a formula for how many floating-point operations it takes to solve an NxN linear equation and combines that with how long it took the computer to execute. So NVidia's tensor instructions actually contribute tens of operations per instruction retired. The prediction is attempting to model when a computer will exceed human computational power, which was estimated by Ray Kurzweil at 20PFLOPS, although it's unclear to me if we was specifically thinking of LINPACK Rmax or some other metric.