4 ms·
The 175B GPT-3 model itself is about 300 GB, in FP16 weights. Server GPUs that consumers can rent capable of running Tensor cores to handle FP16 weights cap ou
by minimaxir 6y ago
The 175B GPT-3 model itself is about 300 GB, in FP16 weights.
Server GPUs that consumers can rent capable of running Tensor cores to handle FP16 weights cap out at 16 GB VRAM. (the recently-announced Ampere-based GPUs cap out at 40 GB VRAM)
tl;dr a local version of the 175B model will not happen any time soon, which is why I really really wish OpenAI would release the smaller models.