4 ms·
Yes, if you can fit it in RAM then the loss isn't too bad if you have the memory pinned and are taking advantage of some of Nvidia's optimizations (which should
by godelski 3y ago
Yes, if you can fit it in RAM then the loss isn't too bad if you have the memory pinned and are taking advantage of some of Nvidia's optimizations (which should be defaulted to on). Inference usually isn't too bad but training is heavier. Things have also massively improved over the last few years. The efficiency also really increases with reduced floating point types and quantized models.