3 ms·
Since you seem knowledgable on this topic, what is it that TPUs do differently than GPUs? Why are they better at inference?
by saiojd 3y ago
Since you seem knowledgable on this topic, what is it that TPUs do differently than GPUs? Why are they better at inference?
- cubefox 3y agoSorry, I actually don't know much about them.
- verdverm 3y agoThey have published various papers and technical reports. The main aim is to make them inhouse and more efficient. Each generation is a little different, like (iirc) v3 is not for training, more for serving at inference time. The use different floating point format and circuits, so they are not good for scientific workloads, iirc again.