3 ms·
But you still need to page in the weights from disk to the GPU at each layer, right?
by ntonozzi 3y ago
But you still need to page in the weights from disk to the GPU at each layer, right?
- juliangoldsmith 3y agoYou should only need the weights for the experts you want to run. The experts clock in at around 400 MB each (based on the 800 GB figure given elsewhere). A 24 GB GPU could fit around 60 experts, so it might be usable with a couple of old M40s.