4 ms·

These chips, like GPUs, are optimized for parallel processing, but they are also designed for lower precision math which requires less memory and fewer transist

by Steko 8y ago

These chips, like GPUs, are optimized for parallel processing, but they are also designed for lower precision math which requires less memory and fewer transistors/operation and hence are considerably more power efficient.

kartickv 8y ago
So it's just a GPU optimised for half-floats and/or integers? I remember reading that it's optimised for matrix calculations. Is that true? If so, what is it about matrix calculations that makes a GPU not the most optimised for this task?
- Steko 8y ago
  Yes it's optimized for 8 bit matrix multiplication. > If so, what is it about matrix calculations that makes a GPU not the most optimised for this task? Well this gets into what is a GPU since now NVIDIA and others are making special tensor GPUs that are also good at this. Most GPUs are optimized for 32 floating point math though, you're probably familiar with "8-bit graphics" being synonymous with 80's NES console quality.
  - kartickv 8y ago
    I see. When I said GPU, I meant a normal GPU, not a special tensor GPU. BTW, what makes tensors / matrices hard for GPUs to handle?
    - Steko 8y ago
      AFAIK GPUs are ok at matrix math, it's just they are typically built for higher precision. AI chips probably have some other advantages but I'm not the best person to speak to them. Here's my simplified understanding: CPUs are optimized for taking a decent amount of data and doing anything with them, especially if the order counts as is often the case with algorithms. GPUs are optimized for taking a lot of data and doing the same few things to a lot of them (shaders, physics). AI chips are optimized at taking a massive amount of data and doing a large number of calculations on all of them. In a neural net each "neuron" and/or "synapse" can represent a function/weight that needs to be calculated across many inputs. In a deep learning set, this could be millions of weights and the more weights you have the more important memory and power efficiency become. When a NN is being looked up into (as opposed to the initial training) it's often in real time (self driving cars, "hey siri", etc.) so the demand to finish all those calculation's faster is even higher. OTOH the accuracy is less important, as neural nets - like people - are good at filtering out noise.

by Steko 8y ago

kartickv 8y ago

So it's just a GPU optimised for half-floats and/or integers? I remember reading that it's optimised for matrix calculations. Is that true? If so, what is it about matrix calculations that makes a GPU not the most optimised for this task?

Steko 8y ago
Yes it's optimized for 8 bit matrix multiplication. > If so, what is it about matrix calculations that makes a GPU not the most optimised for this task? Well this gets into what is a GPU since now NVIDIA and others are making special tensor GPUs that are also good at this. Most GPUs are optimized for 32 floating point math though, you're probably familiar with "8-bit graphics" being synonymous with 80's NES console quality.
- kartickv 8y ago
  I see. When I said GPU, I meant a normal GPU, not a special tensor GPU. BTW, what makes tensors / matrices hard for GPUs to handle?
  - Steko 8y ago
    AFAIK GPUs are ok at matrix math, it's just they are typically built for higher precision. AI chips probably have some other advantages but I'm not the best person to speak to them. Here's my simplified understanding: CPUs are optimized for taking a decent amount of data and doing anything with them, especially if the order counts as is often the case with algorithms. GPUs are optimized for taking a lot of data and doing the same few things to a lot of them (shaders, physics). AI chips are optimized at taking a massive amount of data and doing a large number of calculations on all of them. In a neural net each "neuron" and/or "synapse" can represent a function/weight that needs to be calculated across many inputs. In a deep learning set, this could be millions of weights and the more weights you have the more important memory and power efficiency become. When a NN is being looked up into (as opposed to the initial training) it's often in real time (self driving cars, "hey siri", etc.) so the demand to finish all those calculation's faster is even higher. OTOH the accuracy is less important, as neural nets - like people - are good at filtering out noise.