5 ms·

What even is an "AI chip"? What's the difference to a GPU? As long as nobody can explain that I have big doubts that it whould be more than a GPU+marketing. So

What even is an "AI chip"? What's the difference to a GPU? As long as nobody can explain that I have big doubts that it whould be more than a GPU+marketing. So no big deal if they don't provide one.

lrem 7y ago
I would guess a chip optimised for addition, multiplication and ReLU, not burdened too much by all those other rarely used opcodes.
acidbaseextract 7y ago
Modern GPUs are extremely programmable, but this flexibility isn't that heavily used by neural networks. NN inference is pretty much just a huge amount of matrix multiplication. Especially for mobile applications (most Arm customers), you pay extra energy for all that pipeline flexibility that isn't being used. A dedicated chip will save a bunch of power.
fxj 7y ago
This link gives a nice explanation of the google nn-processor aka TPU: https://cloud.google.com/blog/products/gcp/an-in-depth-look-at-googles-first-tensor-processing-unit-tpu https://cloud.google.com/blog/products/gcp/an-in-depth-look-... It boils down to: CPUs: 10s of cores GPUs: 1000s of cores NNs: 100000s of cores NNs have very simple cores (fused multiply-add and look-up table functions) but can run many of them in one cycle. FTA: Because general-purpose processors such as CPUs and GPUs must provide good performance across a wide range of applications, they have evolved myriad sophisticated, performance-oriented mechanisms. As a side effect, the behavior of those processors can be difficult to predict, which makes it hard to guarantee a certain latency limit on neural network inference. In contrast, TPU design is strictly minimal and deterministic as it has to run only one task at a time: neural network prediction. You can see its simplicity in the floor plan of the TPU die.
endorphone 7y ago
Neural network learning and inference primarily uses matrix multiplication and addition, usually at lower bit depths. You can do this on GPUs to great success and massive parallelism, however the GPU is more generalized than this so it takes more silicon, and more power. With a TPU/neural processor you optimize the silicon to a very, very specific problem, generally multiplying large matrixes and then adding to another matrix. On a GPU we decompose this into a large number of scalar calculations and it massively parallelizes and does a good job, but on a TPU we feed it the matrixes and that's all it's made to do, with very large scale matrix operations, with so much silicon dedicated to matrix operations that it often does it in a single cycle. Another comment mentioned cores and I don't think that's a good way of looking at it, as in most ways a TPU is back to very "few" but hyper-specialized "cores". There is essentially no parallelism in a TPU or neural processor -- you feed it three matrixes and it gives you the result. You move on to the next one.
mruts 7y ago
an AI chip would essentially just be a chip that can do matrix multiplication very quickly plus some addition. Each perceptron (neuron) is just fitting a linear equation, so if we had a chip that could support millions of perceptrons all fitting linear equations (i.e with matrix multiplication), than it would be a huge win compared to a GPUs, which are more general and less efficient for this specific task that dedicated silicon.

What even is an "AI chip"? What's the difference to a GPU? As long as nobody can explain that I have big doubts that it whould be more than a GPU+marketing. So no big deal if they don't provide one.

lrem 7y ago

I would guess a chip optimised for addition, multiplication and ReLU, not burdened too much by all those other rarely used opcodes.

acidbaseextract 7y ago

Modern GPUs are extremely programmable, but this flexibility isn't that heavily used by neural networks. NN inference is pretty much just a huge amount of matrix multiplication. Especially for mobile applications (most Arm customers), you pay extra energy for all that pipeline flexibility that isn't being used. A dedicated chip will save a bunch of power.

fxj 7y ago

This link gives a nice explanation of the google nn-processor aka TPU: https://cloud.google.com/blog/products/gcp/an-in-depth-look-at-googles-first-tensor-processing-unit-tpu https://cloud.google.com/blog/products/gcp/an-in-depth-look-... It boils down to: CPUs: 10s of cores GPUs: 1000s of cores NNs: 100000s of cores NNs have very simple cores (fused multiply-add and look-up table functions) but can run many of them in one cycle. FTA: Because general-purpose processors such as CPUs and GPUs must provide good performance across a wide range of applications, they have evolved myriad sophisticated, performance-oriented mechanisms. As a side effect, the behavior of those processors can be difficult to predict, which makes it hard to guarantee a certain latency limit on neural network inference. In contrast, TPU design is strictly minimal and deterministic as it has to run only one task at a time: neural network prediction. You can see its simplicity in the floor plan of the TPU die.

endorphone 7y ago

Neural network learning and inference primarily uses matrix multiplication and addition, usually at lower bit depths. You can do this on GPUs to great success and massive parallelism, however the GPU is more generalized than this so it takes more silicon, and more power. With a TPU/neural processor you optimize the silicon to a very, very specific problem, generally multiplying large matrixes and then adding to another matrix. On a GPU we decompose this into a large number of scalar calculations and it massively parallelizes and does a good job, but on a TPU we feed it the matrixes and that's all it's made to do, with very large scale matrix operations, with so much silicon dedicated to matrix operations that it often does it in a single cycle. Another comment mentioned cores and I don't think that's a good way of looking at it, as in most ways a TPU is back to very "few" but hyper-specialized "cores". There is essentially no parallelism in a TPU or neural processor -- you feed it three matrixes and it gives you the result. You move on to the next one.

mruts 7y ago

an AI chip would essentially just be a chip that can do matrix multiplication very quickly plus some addition. Each perceptron (neuron) is just fitting a linear equation, so if we had a chip that could support millions of perceptrons all fitting linear equations (i.e with matrix multiplication), than it would be a huge win compared to a GPUs, which are more general and less efficient for this specific task that dedicated silicon.