4 ms·
How much vram is need to run this?
by mithrilskeleton 2y ago
How much vram is need to run this?
- MacsHeadroom 2y ago80GB in 4bit. But because it only activates one expert at a time, it can run on a fast CPU in reasonable time. So 96GB of DDR4 will do. 96GB of DDR5 is better.
- Me1000 2y agoWizardLM-2 8x22b (which was a fine tune of the Mixtral 8x22b base model) at 4bit was only 80GB.