3 ms·
That is true for single user/light inference only. For training and batch inference you can get compute bound fast enough.
by imjonse 2y ago
That is true for single user/light inference only. For training and batch inference you can get compute bound fast enough.
- saagarjha 2y agoThat really depends on what you're doing. Trying to feed a tensor core is pretty hard–they're really fast.