3 ms·
What’s the minimum single GPU that’ll work for the smallest model?
by yumraj 4y ago
What’s the minimum single GPU that’ll work for the smallest model?
- downvotetruth 4y ago3060 12GB
- zargon 4y agoThis reddit post says that the 7B model consumes about 9.7GB of VRAM (using int8). I'm sure very soon people will add support for using system RAM as swap space, which will allow you to use it on an 8GB card, though with a fairly hefty performance penalty. https://www.reddit.com/r/MachineLearning/comments/11h3p2x/d_facebooks_llama_leaks_via_torrent_file_in_pr/jb0wots/ https://www.reddit.com/r/MachineLearning/comments/11h3p2x/d_...