3 ms·
I've heard credible claims that GPUs these days (esp. TPUs) have lower latency for big models than CPUs. I haven't really investigated, but I could see it happe
by singhrac 6y ago
I've heard credible claims that GPUs these days (esp. TPUs) have lower latency for big models than CPUs. I haven't really investigated, but I could see it happening if you give the TPU a huge L1 cache or something.
- someguydave 6y agoPerhaps for large calculations? Otherwise the PCI transfer delay would be a big latency hit?
- adamnemecek 6y agoYeah until TPUs can directly communicate with the sound card, it sounds slow.