3 ms·
TPUs do compete with GPUs for ML tasks, so yes, this is evidence that GPUs are winning. The only alternative I could imagine is that TPUs will "win" at superco
by cubefox 4y ago
TPUs do compete with GPUs for ML tasks, so yes, this is evidence that GPUs are winning.
The only alternative I could imagine is that TPUs will "win" at supercomputers exclusivity aimed at inference (as opposed to training). Since TPUs excel at inference. The question is how much ML compute is used for inference as opposed to training. Not much, I guess, otherwise something like TPUs would be more popular.
- sebzim4500 4y agoI've heard estimates that the amount of compute used to train GPT-4 is equivalent to 8 months of usage and most models are used much less than GPT-4 is, although I guess they are also easier to train.
- hoschicz 4y agoof "usage"? I never bought that claim as it's not clear what usage they mean – on 100x8 months or 10000x8 months?
- saiojd 4y agoSince you seem knowledgable on this topic, what is it that TPUs do differently than GPUs? Why are they better at inference?
- cubefox 4y agoSorry, I actually don't know much about them.
- verdverm 4y agoThey have published various papers and technical reports. The main aim is to make them inhouse and more efficient. Each generation is a little different, like (iirc) v3 is not for training, more for serving at inference time. The use different floating point format and circuits, so they are not good for scientific workloads, iirc again.