3 ms·
We use the 3090 for testing and development, because it is "similar enough" to a V100. For deployment, we were evaluating A100 and V100. (You're not allowed to
by fxtentacle 4y ago
We use the 3090 for testing and development, because it is "similar enough" to a V100. For deployment, we were evaluating A100 and V100. (You're not allowed to put a 3090 into a datacenter.) Since it's a 900 mio parameter model, T4 and A2 don't have enough RAM, I can barely train this thing on a 3090 ^_^
- david-gpu 4y agoOnce models are trained, it is not too difficult to compress them. Have you done any of that? You can start with things like switching the weights and activations to fp16. A100 also supports BFloat16, which is even better and should work right out of the box.