5 ms·
The answer is yes if you have a 24GB GPU. Just wait for 4bit quantization. Or watch Tim Dettmers, who is releasing code to run Mixtral 8x7b in just 4GB of RAM.
by MacsHeadroom 3y ago
The answer is yes if you have a 24GB GPU. Just wait for 4bit quantization.
Or watch Tim Dettmers, who is releasing code to run Mixtral 8x7b in just 4GB of RAM.