5 ms·
Can you provide a source on the memory starvation for the TPUs? Plus are we talking generation 1 or 2 or both? Did not seem to be an issue in the paper with
by jacksmith21006 9y ago
Can you provide a source on the memory starvation for the TPUs? Plus are we talking generation 1 or 2 or both?
Did not seem to be an issue in the paper with the first generation.
https://cloud.google.com/blog/big-data/2017/05/an-in-depth-look-at-googles-first-tensor-processing-unit-tpu https://cloud.google.com/blog/big-data/2017/05/an-in-depth-l...
An in-depth look at Google's first Tensor Processing Unit (TPU ...
- baybal2 9y agoThis is what was intensively discussed by first trial users, and later admitted by google tpu team themselves on hot chips 29 Even with monsterous HBM2 memory, they still have it. It is probably hard to predict what matrix set to prefetch when you deal with a neural net. So you have cache misses there too
- pjscott 9y agoIn the paper on the first-generation TPU, in section 7. They estimate that there would have been impressive gains in speed, both absolute and per-watt, if they'd had enough design time to give it more memory bandwidth: https://drive.google.com/file/d/0Bx4hafXDDq2EMzRNcy1vSUxtcEk/view https://drive.google.com/file/d/0Bx4hafXDDq2EMzRNcy1vSUxtcEk...