4 ms·
This is what was intensively discussed by first trial users, and later admitted by google tpu team themselves on hot chips 29 Even with monsterous HBM2 memory,
by baybal2 9y ago
This is what was intensively discussed by first trial users, and later admitted by google tpu team themselves on hot chips 29
Even with monsterous HBM2 memory, they still have it.
It is probably hard to predict what matrix set to prefetch when you deal with a neural net. So you have cache misses there too