4 ms·
Why would a user need a 12gb gpu to run the game locally? The deep learning model is already trained, and I can’t imagine one needs a gpu to just evaluate the m
by clircle 7y ago
Why would a user need a 12gb gpu to run the game locally? The deep learning model is already trained, and I can’t imagine one needs a gpu to just evaluate the model.
- minimaxir 7y agoIt's a big model.
- Dylan16807 7y agoIt's a 6GB download. Mid-high end GPUs are 8GB. So what makes it need 12GB? How specifically is that memory used up by various types of decompression, intermediate calculations, etc.
- dodobirdlord 7y agoI believe that the 1.5B model weights take up the 6GB themselves. Presumably f32 weight values? Since they are all needed for an evaluation pass they will eat up 6GB on the GPU off the bat. Not too surprising that everything else can't fit in 2GB, since that's going to have to fit the entire model architecture and all intermediary values.
- Dylan16807 7y agoWhat exactly are these intermediate values composed of that makes them notable in size compared to the model itself? Are there resources I should read for how a model like this executes? Will this model work with half precision weights? Is it very awkward to use "brain" 16 bit floats?
- dodobirdlord 7y agoThe model has 1.5 billion parameters, so requiring a beefy GPU to evaluate the model is unsurprising.
- Filligree 7y agoYou can use your CPU, certainly. It'll take a couple seconds per reply.