3 ms·

As I understand it the main advantage of the TPU is that it is built around an 8 bit pipeline. It's counter-intuitive to me, but apparently many deep learning N

As I understand it the main advantage of the TPU is that it is built around an 8 bit pipeline. It's counter-intuitive to me, but apparently many deep learning NN can be flattened down to layers with only 256 states and still perform really well.

gok 9y ago
The second generation TPU is actually FP16 based.
Symmetry 9y ago
That's one part but doing sequencing in hardware is another big factor. And I expect they're playing tricks to reduce the read pressure on their register files.